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I. INTRODUCTION 
A. OVERVIEW 

1. Background 

In the last half century, Naval Aviation has reduced the Class A Flight Mishaps 
(FM) rate significantly. In fact, the Class A FM rate was reduced by 50 percent in each 
decade from 1950 to 2000 (see Figure 1). Despite this success, the cost of mishaps, in 
terms of lives and assets, remains too high. Furthermore, human error, attributable to 
four out of every five mishaps, has remained constant (Nutwell & Sherman, 1997). In 
response to a 1996 F-14 Tomcat FM, which was largely attributable to human error, a 
Human Factors Quality Management Board (HFQMB) was initiated to study human error 
in Naval Aviation mishaps. The HFQMB’s charter (1997) was to reduce the mishap rate 
by identifying systematic improvements in processes and systems that affect human 
performance and to institutionalize continual improvements in these areas. The HFQMB 
is made up of Flag Officers, senior operational officers, and a cross section of 
Commanding Officers to seasoned naval aviators from all communities. Their objective 
was to reduce the Class A FM rate by half within three years, and by 75 percent within 10 
years (HFQMB, 1997). 

The HFQMB‘s strategy entailed a three-pronged approach: (1) Mishap Data 
Analysis (MDA), (2) Organizational Benchmarking (OB), and (3) Command Safety 
Assessment (CSA). MDA studies historical records to determine established trends of 
human error in aviation mishaps. The Human Factors Analysis and Classification 
System (HFACS) was developed to accomplish this task. By examining and classifying 


past mishaps, prevailing human errors are prioritized and targeted for prevention. The 


second phase, OB, involved the observance of other organizations, both military and 
civilian, for procedures and practices that might be useful in helping reduce the 
occurrence of human error. To this end, Naval Aviation adopted Organizational Risk 
Management, which associates conditions with risk and decision making methodology to 
reduce the occurrence of aviation mishaps, directly from the U. S. Army (Department of 
the Navy, 1997). CSA was developed at the Naval Postgraduate School by Civarelli and 
Figlock (1997) as a tool to measure the safety climate in a squadron from an aircrew 
perspective. It solicits opinions and attitudes towards procedures and practices to 
determine the effect of organizational and supervisory issues on flight safety (Nutwell & 


Sherman, 1997). 
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Figure 1: FY 1950-1999 Naval Aviation Class A Flight Mishap Rates 
(From School of Aviation Safety, 1999) 


In January 1999, the Vice Chief of Naval Operations extended the short-term goal 
of 50 percent reduction in Class A FM mishap rate from the end of FY 1999 to the end of 


FY2000. To meet this goal, the scope of the HFQMB needed to expand to include 


human error in maintenance mishaps. Historically, approximately one in every five Class 
A FMs contains maintenance error. By expanding the focus to include maintenance 
error, it was believed that the 50 percent reduction objective could be met. An identical 
three-prong approach that had been used for aircrew error was utilized for maintenance 
error. 

2. Maintenance Mishap Data Analysis 

HFACS was developed as a framework for examining aircrew and supervisory 
error in Class A FMs. It concurs with the Naval Aviation Safety Program’s 
(OPNVAVINST 3750.6) notion of multiple causal factors, chain of events, and human 
factors. A maintenance extension (ME) was added to HFACS to meet the HFQMB’s 
expanded scope and was used to examine incidents and injuries (Schmidt, Figlock & 
Teeters, 1999), and major to minor mishap data (Schmidt, Schmorrow & Figlock, 2000). 
HFACS-ME was adopted for the upcoming revision of the Naval Aviation Safety 
Program. Furthermore, a laptop prototype tool, Maintenance Extension Information 
Management System (MEIMS) was developed to allow users to collect, catalog, collate 
and analyze mishap data, to better identify trends and for safety training. MEIMS, in 
limited usability testing, was determined to be a valuable tool that will provide the fleet 


great benefit (Wood, 2000). 


B. PROBLEM STATEMENT 
To meet the objective of a 50 percent reduction in Class A FMs, both aircrew and 
maintenance error needed to be targeted. The HFACS-ME taxonomy and MEIMS were 


developed to better analyze maintenance error. While proven useful in a limited usability 


study, MEIMS 1s not without limitations. Users found the tool to be somewhat difficult 
to navigate, was lacking in assistance functionality and its interface needed to be more 
user-friendly. Additionally, it was determined that its intended users (e.g., squadron 
maintenance and safety personnel, squadron safety officers, Aircraft Mishap Board 
members, Naval Safety Center analysts, etc) would benefit from a training tutorial prior 
to accessing the tool (Wood, 2000). This study will further develop MEIMS based upon 
the initial usability study. It is believed that the fleet user, with proper training, should be 
able to easily access the tool and obtain useful information, which can then be used in 
training, hazard identification and trend analysis, to prevent future mishaps. 
This thesis investigates the following questions: 
1. What query capabilities are required by the users of the information 
system to best identify problem areas and trends? 
2. What additions will make the information system more user-friendly, 
allowing efficient and effective access to the system’s capabilities? 
3. What kind of training, in the form of a tutorial, would best prepare users to 
efficiently and effectively access the information system’s full 


capabilities? 


C. PURPOSE 

The intent of this study is to refine and evaluate the existing safety information 
management system, which uses the HFACS-ME taxonomy as its basis, to better 
facilitate data collection, organization, query, analysis and reporting of maintenance 


personnel errors that contribute to Naval Aviation mishaps, equipment damage and 


personnel injury. Included are theoretical approaches that examine mishaps involving 
human error: Heinrich’s “Domino” Theory, Edwards’ “SHEL” Model, and Reason’s 
“Swiss Cheese” Model. These models not only help identify the unsafe actions which 
cause mishaps, but they also connect the conditions underlying the mishap. HFACS-ME 
is a composite derivative of these taxonomies. Although there is no universally accepted 
method of accident investigation (Benner, 1975), some standardized aircraft accident 
investigation procedures have been adopted by most agencies throughout the world 
(Diehl, 1991). Using HFACS-ME as its basis, MEIMS will significantly improve access 
and analysis of mishap data. 

The results of this study will (1) improve and enhance the MEIMS prototype 
tool’s usability so that data can be easily queried and analyzed, (2) creation of a tutorial 
for fleet customers that will enhance their ability to not only access the data base, but to 
efficiently and effectively use this information to prevent future mishaps, and (3) study 
the performance of two groups of participants, with only one group receiving the tutorial, 


to determine the effectiveness of the tutorial for fleet users. 


D. SCOPE AND LIMITATIONS 

MEIMS will be revised to include a users’ tutorial. Fleet personnel, primarily 
Aviation Safety Officers, will test MEIMS to determine its usability and the benefits of 
the help and tutorial functionality. The fully developed MEIMS tool will be used 
primarily by Naval Aviation squadrons, but may have some crossover use by other 


military branches and civilian airlines. Only maintenance related mishaps caused by 


human error are considered. No material failure factors or maintenance related hazard 


reports or personnel injuries not related to a mishap are included. 


E. DEFINITIONS 
This study uses the following definitions: 

Aircraft Mishap Board: Group of officers appointed to investigate and report on 
an aviation mishap. 

Aviation mishap rate: Number of aviation mishaps per 100,000 flight hours. 

Aviation Safety Officer: Principal advisor to Naval Aviation squadron 
commanding officers on all aviation safety matters. 

F-14 Tomcat: U. S. Navy’s swing-wing, supersonic fighter aircraft with air-to-air, 
air-to-ground, and reconnaissance capability. 

Fleet Logistics Support Wing: U. S. Navy reserve air wing comprised of transport 
aircraft. 

HEFACS: Human Factors Analysis and Classification System designed to help 
analyze Naval Aviation mishaps focusing on aircrew error. 

HFACS-ME: Human Factors Analysis and Classification System--Maintenance 
Extension adaptation to classify causal factors that contribute to maintenance mishaps. 

HFOMB: Human Factors Quality Management Board established by Naval 
Aviation senior leadership to reduce human error involvement in Naval Aviation Class A 
flight mishaps. 

MEIMS: Maintenance Error Information Management System, a prototype tool 


developed to collect, catalog, collate and analyze mishap data. 


Mishap: A naval mishap is an unplanned event or series of events directly 
involving naval aircraft, which result in $10,000 or greater cumulative damage to naval 
aircraft, other aircraft, property, or personnel injury. 

Mishap Categories: Naval aircraft mishap categories are defined below: 

Flight Mishap (FM): Those mishaps in which there was $10,000 or 
greater DOD aircraft damage or loss of a DOD aircraft, and intent for flight for 
DOD aircraft existed at the time of the mishap. Other property damage, injury, or 
death may or may not have occurred. 

Flight Related Mishap (FRM): Those mishaps in which there was less 
than $10,000 DOD aircraft damage, and intent for flight (for DOD aircraft) 
existed at the time of the mishap, and $10,000 or more total damage or a defined 
injury or death occurred. 

Aircraft Ground Mishap (AGM): Those mishaps in which no intent for 
flight existed at the time of the mishap and DOD aircraft loss, or $10,000 or more 
aircraft damage, and/or property damage, or a defined injury or death occurred. 
Mishap Severity Class: Mishap severity based on injury and property damage. 

Class _A: A mishap in which the total cost of property damage 
(including all aircraft damage) is $1,000,000 or greater; or a naval aircraft is 
destroyed or missing; or any fatality or permanent total disability occurs with 
direct involvement of naval aircraft. 

Class _B: A mishap in which the total cost of property damage 


(including all aircraft damage) is $200,000 or more, but less than $1,000,000 


and/or a permanent partial disability, and/or the hospitalization of five or more 
personnel. 
Class C: A mishap in which the total cost of property damage 

(including all aircraft damage) is $10,000 or more but less then $200,000 and/or 

injury results in five or more lost workdays. 

Naval Aircraft: Refers to U.S. Navy, Naval Reserve, and Marine Corps aircraft. 

OPNAVINST 3750.6: The Naval Aviation Safety Program: U.S. Navy instruction 
outlining Naval Aviation’s safety program. 

QRM: Operational Risk Management. A decision making tool to increase 
effectiveness by anticipating and reducing hazards, thus increasing the probability of a 


successful mission. 


F. ORGANIZATION OF STUDY 

Chapter II contains a literature review on the development of a prototype to 
identify human error involvement and patterns in aviation maintenance mishaps. The 
methods used in this study are discussed in Chapter III. The results of this study are 
presented in Chapter IV. Finally, Chapter V contains conclusions, findings, and 


recommendations. 


Il. LITERATURE REVIEW 

A. OVERVIEW 

The literature studied encompasses human error, maintenance error in aviation, 
error classification and analysis. It includes textbooks, research articles, and masters 
thesis pertaining to: (1) human error theories, and their relation to maintenance related 
aviation mishaps, (2) accident information management, and (3) design and usability 
assessment of software tools. This information provides the basis for the continuing 
development of the maintenance error analysis and reporting prototype tool. 
While ongoing efforts to reduce the number of Class A mishaps in Naval Aviation are to 
be applauded, there is potential for further improvement, especially in the area of human 
error. When examining mishaps involving human error there are numerous theoretical 
approaches from which to choose (Goetsch, 1996). Some are founded in industrial 
safety; others are viewed from a complex systems perspective, emphasizing human 
factors and operator error. Some approaches use models drawn from the domain of 
preventive medicine that employ epidemiological factors to analyze accidents, while 


other models utilize a combination of these approaches (Schmidt, 1998). 


B. ACCIDENT DATA AND MANAGEMENT 

1. Investigation and Reporting 

Diehl’s (1991) three-stage TRO of accident investigation and prevention focuses 
on human performance and systems safety considerations (see Figure 2). Diehl’s stages 
are Accident Generation, Accident Investigation Process, and Preventive Measures. The 


purpose of the first stage, Accident Generation, is the identification of hazards. Hazards 


are circumstances or conditions which have the potential to lead to an incident (near- 
accident) or even an accident. The second stage. Accident Investigation Process, 
includes the collection, analysis, and review of accident data and the focus of this review. 
In aircraft accident investigations, procedure calls for a fact finding body to investigate 
the accident to determine what happened, and subsequently what caused accidents. 
Lastly, a final report is made, which details causes of accidents and makes 


recommendations to prevent reoccurrence (Diehl, 1991). 
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Figure 2: Accident Generation, Investigation, and Prevention Elements 
(After Diehl, 1989) 

The final stage, Prevention Measures, details the methods used to avoid future 
accidents. There are four categories of accident-prevention measures: (1) eliminating 
hazards and risks, (2) incorporating safety features (3) providing warning devices, and 
(4) establishing procedural safeguards. In ascending order, these measures are more 


effective, but are also more expensive and more difficult to attain (Diehl, 1991). 


2. Accident Prevention 

In the early 1900s, accident investigations were based on the notion people 
commit unsafe acts that lead to incidents (Heinrich, 1959). This blame-based practice 
deterred systematic accident prevention until the 1950’s, when systems engineering 
emerged from of the U.S. military’s large-scale weapons program. Applied to accident 
prevention, systems engineering focuses on the strengths and limitations of components, 
including the human element, as an integral part of the system (Heinrich, Petersen & 
Roos, 1980). Presently, up to 90 percent of accidents can be attributable to human error 
in a variety of government, military and industrial settings (Heinrich, Petersen, & Roos, 
1980; Hale & Glendon, 1987; School of Aviation Safety, 2000), so analyzing the human 
element can vastly improve the understanding of systems and why they fail. 

Operational Risk Management (ORM), an outgrowth of systems engineering, is a 
tool that the armed forces employed to decrease accidents. ORM focuses on the 
identification of hazards during mission planning to effect control measures which reduce 
the associated risk. It is especially effective in highlighting human factor hazards (DON, 
1997). The U.S. Army's aviation branch achieved record low accident rates in 1995 and 
1996, largely attributable to their employment of ORM (DOA, 2000). Since 19977, the 
other military services institutionalized ORM into their operational doctrine (School of 
Avaition Safety, 2000). 

3. Data Management 

To be useful in the prevention of accidents, relevant data must be collected and 
properly archived for future reference. Coding the data and the use of data bases have 


become universally accepted methods for this task. The National Safety Council 


established a numerical code system relating to each cause factor (National Safety News, 
1975). This method of coding simultaneously made referencing for specific cause factors 
much simpler, and also provided more efficient analysis capability by focusing on 
specific causes rather than accidents as a whole. Additionally, it is essential to also 
include “near-miss” incident data, as their circumstances are as likely to cause an 
accident in the future. Computer analysis tools can significantly aid reviews of mishap 
histories (Grimaldi & Simonds, 1984). To be truly effective in reducing future accidents 
and incidents those tools must organize and tabulate the data logically, and present it for 


analysis in easily accessible, user friendly formats. 


C. ACCIDENT CAUSATION THEORIES 

There are numerous theoretical approaches to examine mishaps involving human 
error (Goetsch, 1996). Three of the most recognized approaches are Heinrich’s Domino 
Theory, Edwards’ SHEL Model and Reason’s Swiss Cheese Model. Despite differences 
in approach and originating source, these theories compliment each other and provide a 
solid foundation for analyzing human error in avaition mishaps. 

1. Heinrich’s “Domino” Theory 

Considered to be the original accident theory (Goetsch, 1996), it details a linear 
five step sequence of related factors (chain of events) that lead to an actual mishap. Bird 
(1980) also held this view. The two central principles of the Domino Theory are (Bird, 
1980): (1) accidents are caused by the combined actions of the preceding factors, and (2) 
removal of the middle factor (unsafe act or condition) will negate the actions of the 


preceding factors and thus prevent accidents and injuries (see Figure 3). 
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Figure 3: The Five-Step Domino Sequence (Bird, 1980) 


Heinrich’s original five-factor model (Heinrich, Petersen, & Roos, 1980), can be 


paraphrased as follows: 


L 


Lack of Control: This is a management issue where the emphasis is placed 
on the control exercised in a situation for an array of factors. 
Basic Cause(s): This identifies the origin(s) of the causes and includes 


aspects such as human factors, environmental factors, or job-related 


factors. 


Immediate Cause(s): This includes substandard practices and conditions 


that are symptoms of the basic causes. 


Incident: This typically involves contact with the hazard, and for example, 


results in a fall or impact with moving objects. 


Personal Injury and Property Damage: This includes lacerations, fractures, 


death, and material damage. 


Each step preempts the next, causing it to occur; much the same way as one 


domino falling causes the next domino in sequence to fall as well. Removal of the 


factors that comprise any of the first three “dominos” will effectively intervene to prevent 


the accident. 
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2. Edwards’ “SHEL” Model 

The “SHEL Model” (see Figure 4) of system design was developed in the early 
1970’s to provide a better way to evaluate failures in human-machine systems (Hawkins, 
1992). It identifies and defines four system dimensions: Software, Hardware, 
Environment, and Liveware. The “SHEL Model” provides a method to describe systems, 
identify potential areas for concern within a system, and proposes a general framework 
for accident investigation. Edwards (1988) defines SHEL concepts as follows: 
Software: the rules, regulations, laws, orders, standard operating procedures, customs, 
practices, and habits that govern the manner in which the system operates and in which 
the information within it is organized. Software is typically a collection of documents. 


1. Hardware: the buildings, vehicles, equipment, and materials of which the 
system is comprised. 

2. Environmental conditions: the physical, economic, political and social 
factors within which the software, hardware, and liveware operate. 

3. Liveware: the human beings involved with the system. 

According to the “SHEL Model,” the system will fail when a disconnect occurs in 
any one of the four dimensions, or in the connections between them. People are rarely 
the sole cause of accidents (Edwards, 1988), but rather, accidents are caused by the 
interaction of several factors (Shappell & Wiegmann, 1997). This multi-factor theory 


was a Significant departure from the commonly held belief that accidents were caused by 


single events. 
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Figure 4: SHEL Model of System Design (Hawkins, 1993) 

3. Reason’s “Swiss Cheese’ Model 

Reason’s “Swiss Cheese” Model employs a vertical association human factors 
approach to a collection of factors that eventually lead to an accident. It differentiates 
between two types of errors: 1) active failures, whose effects are felt immediately, and 
2) latent conditions, whose effects may lie dormant until triggered later, usually by other 
mitigating factors (Reason, 1990). Latent conditions “‘set the stage” for an accident while 
active failures tend to be the catalyst for the accident to finally occur. The model can be 
thought of as slices of Swiss cheese lined up, with each vertical slice representing a 
defense layer (e.g. training, good management, teamwork, etc.) and each hole 
representing an active failure or latent condition in that defense (see Figure 5). Should a 
situation where holes line up come to pass, an accident will occur. 

This is a dynamic model with each defensive layer coming in and out of 
prominence according to situational characteristics (Reason, 1990). An event may occur 
in one of three levels: (1) person-unsafe acts, (2) workplace-error provoking conditions, 
and (3) organization-error establishing conditions. Organizational factors, in which 
strategic decisions and associated processes (budgeting, forecasting, resource allocation) 


are initiated, provide the starting point for an accident (Reason, 1997). These processes, 
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influenced by corporate culture, are distributed throughout the organization to individual 
workplaces. Corporate processes, evidenced as inadequate staffing, time pressures, 
equipment, training and working conditions, combine with the natural proclivity to 
commit errors and/or violations, to result in unsafe acts. Few of these acts, however, 


actually create holes in the defense layers en route to becoming an accident. 
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Figure 5: Reason’s Swiss Cheese Model (Naval Safety Center, 1996) 


D. HUMAN FACTORS ANALYSIS & CLASSIFICATION SYSTEM (HFACS) 
Efforts to analyze aircrew error in Naval Aviation mishaps led to the development 
of HFACS (Shappell & Wiegmann, 1997). It began as a restructuring and expansion of 
the Swiss Cheese Model, but it also includes features of Heinrich’s Domino Theory and 
Edwards’ SHEL Model. The HFACS taxonomy recognizes the set of conditions within 
which system operators perform, accounts for interrelationships among conditions, and 
provides an overall sense of order or implied hierarchy. It identifies both active failures 


and latent conditions within four categories (DON, 2000): (1) unsafe acts; (2) pre- 
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conditions for unsafe acts; (3) unsafe supervision, and (4) organizational influences. This 
classification is then used to target the most appropriate intervention. 

While Reason’s “Swiss Cheese” model is widely held as integral to explaining 
accidents, it does not provide a means of delineating precursors to accidents (Shappell &, 
Weigmann 1997). Although modern analysis systems are heading in the right direction 
regarding human factors investigation, more effort is needed. There exists a requirement 
for a framework that allows for the explanation of accident causation involving human 
error to include not only the causes of those errors, but also an explanation of those 
causes. It is through such efforts that further occurrences of the same type of accident 
can be avoided. 

1. Maintenance Mishaps 

The aviation industry has, in recent years, begun to apply human error theory to 
maintenance-related mishaps with the intent of modernizing accident investigations 
(Marx, 1998). Previously, it was believed that every error could be traced back to a basic 
set of actions and associated conditions that precipitated the error (Goetsch, 1996). This 
simplistic view did not allow for errors to have multiple causes, as most do. 

Additionally, traditional investigation techniques, while appropriate for identifying the 
causes of equipment failures, did not have the same success with human error-related 
accidents (O'Connor & Bacchi, 1997). Typically, investigations would effectively end 
when the cause pointed to human error, with no effort expended in attempting to explain 
why the error occurred. 

In a review of investigation and analysis systems for aircraft maintenance error, 


the need for human factors investigation and reporting was identified (Marx, 1998). 


ey 


Although human factors investigation methods are acknowledged as critical to 
understanding why people make certain mistakes, they have not been widely adopted. 
The proper investigation of human factors, however, 1s vital to accident prevention 
(Harle, 1994). The lack of thorough human factors investigation was caused by the 
tendency to place blame, inability to see through proximate causes to underlying causes, 
and an over-emphasis on static factors such as who, what and when (Marx, 1998). 

2. HFACS-ME Maintenance Extension 

In light of the above requirements, the HFACS taxonomy was adapted, and 
augmented to classify factors that lead to maintenance mishaps (Schmidt, 1996). The 
Maintenance Extension (ME) of the original HFACS taxonomy focuses on causation 
factors particular to the maintenance environment. These aspects and their relevant sub- 
section classifications are given in a manner similar to Reason’s model. In the HFACS- 
ME model, a maintainer’s performance is seen as being influenced by a series of latent 
conditions (supervisory, maintainer and working conditions) that can lead to an Unsafe 
Maintainer Act, which in turn can lead to a mishap, injury or an unsafe maintenance 


condition (see Figure 6). 





Figure 6: HFACS Maintenance Extension (HFACS-ME) 
(DON, 2000) 


HFACS-ME consists of four broad human error categories, three latent: 
Management Conditions (e.g. inadequate supervision), Maintainer Conditions (e.g. 
preparation/training), Working Conditions (e.g. lighting/light), and the fourth Maintainer 
Acts (e.g. skill error), is active (DON, 2000). The three orders of maintenance error 
(first, second, and third) reflect a decomposition of the error types from a macro to a 
micro perspective. Each successive order provides for greater granularity, serving the 
respective purposes of identifying problem areas, prioritizing potential targets, and 


tailoring intervention strategies (see Table 1). 


Table 1: HEACS Maintenance Extension Categories (DON, 2000) 
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3. Maintenance Error Information Management System (MEIMS) 
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The HFACS-ME taxonomy was used as the framework for the development of a 
laptop prototype database tool, MEIMS. It is intended for fleet users to collect, catalog, 
collate and analyze mishap data, and to better identify trends and for safety training. In 
limited usability testing, MEIMS was proven to be effective, yet was lacking in some 


areas. Specifically MEIMS was somewhat difficult to navigate, was lacking in assistance 
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functionality and it’s customer interface needed to be more user-friendly. The benefit of 
user training was also indicated. To fully reach its potential, however, MEIMS needs 


design refinements and more rigorous usability testing (Wood, 2000). 


E. USER INTERFACE 

The proliferation of computer systems into ever increasing environments 
highlights the need for effective and easy to use interfaces. Clearly, the user interface is 
the most important factor in determining the success or failure of a software application 
(Liu, 1997). The design objectives of user interface include effectiveness, efficiency, 
comfort and safety. The content of an interface should abstract critical features of the 
target system (Norman, 1983; Vicente & Rasmussen, 1992), be consistent with human 
conceptual and cognitive characteristics (Carroll, Mack & Kellog, 1988; Wickens, 1992), 
be compatible with human response tendencies (Myers, 1991; Shneiderman, 1983), and 
should help users effectively achieve their intended impact on the target system 
(Sheridan, 1984; Shneiderman, 1992). 

Williges, Williges & Elkerton (1987) examined design objectives from user's 
perspective and generated a list of seven general design principles as a basis for 
developing specific design objectives. The principles are (1) consistency, (2) 
compatibility, (3) memory, (4) structure, (5) feedback, (6) workload, and (7) 
individualization. According to these principles, a well designed user interface should 
maintain the same style of interaction within itself and similar applications (consistency), 
provide easily understandable interface symbols and icons to minimize the demand for 


information recoding (compatibility), minimize the demand on the user's short-term 
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memory (memory), help users understand the structure of the system (structure), provide 
feedback and error-correction mechanisms (workload), and accommodate individual 
differences (individualization). 

From an engineering point of view, Shneiderman (1992) detailed four essential 
classes of goals for interface design. They are (1) functionality to accomplish all 
assigned tasks, (2) foolproof reliability, availability, security and data integrity, (3) 
standardized styles and interface features to ensure portability over multiple applications, 
and (4) development should maintain schedule and remain within budget. The goal of 
interface design not only includes assisting the user in accomplishing their task of 
controlling or influencing a target system, it should also help the user understand the 
system. 

In most cases, the target system and the user interface are usually designed by 
separate individuals, who are often from different groups or organizations. The design 
activities may take place at different places and at different times, making effective 
communications between the two groups especially challenging. The two design teams 
may have different objectives, constraints and conceptual bases. Interface designers must 
establish early contact with not only future users of the system, but also designers of the 
system so that both the "user model" and the "design model" can be clearly identified and 
correctly reconciled. Interface designers must recognize and overcome these competing 
models to successfully bridge the gap between the user’s mental model and the system 


designer’s conceptual model of the same target system (Norman, 1983). 
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F. USABILITY TESTING 

Although user testing with real users is the most fundamental usability method, 
other usability methods can serve as good supplements to gather additional information 
or gain usability insight at a lower cost. User testing, combined with heuristic evaluation 
and other usability inspection methods, can provide better efficiency (Nielsen, 1997). 
Heuristic evaluation consists of finding usability problems in a design by contrasting it 
with a list of established usability principles. Though this method works very well and 
produces fast evaluation results, there are always some issues that will be left 
undiscovered until actual user testing (Nielsen, 1997). In all testing, issues of reliability 
and validity need serious consideration. Reliability considers whether one would get the 
same result if a test were repeated and validity considers whether the result actually 
reflects the usability issues one wants to test. 

1. Reliability 

Huge individual differences between test users presents a reliability problem with 
usability testing. It is not uncommon for the best user to be 10 times faster than the 
slowest user, or the top 25 percent of the users to be twice as fast as the slowest 25 
percent of the users (Egan, 1988). Due to this phenomenon, one cannot conclude much 
from observing a user A using interface X performing a certain task 40 percent faster 
than user B using interface Y. User B could very well just happen to be slower than user 
A. If the test were repeated with users C and D, the result could easily be exactly 
opposite. Often decisions must be made on the basis of fairly unreliable data, which 
while suspect, is better than having no data at all. Standard statistical tests should be used 


assess test results and thereby indicate the significance and reliability. Although a 0.05 
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significance level is often used for research studies, for practical development purposes 
an 0.20 significance level might be more attainable due to budget and time restraints 
(Nielsen, 19977). 

2. Validity 

While reliability can be addressed with statistical tests, a high level of validity 
requires full understanding of the test method, as well as common sense. Typical validity 
problems include using the wrong users, giving them the wrong tasks, or not including 
time restraints of social influences. Confounding effects may also lower the validity of a 
usability test. An example would be testing the move from a character based user 
interface to a graphical user interface for an application. If, while running this test, the 
two competing systems were running on screens set to different pixel graphic displays, 
the test results may give a comparison between the two size screens rather than the 
intended character-based versus graphical user interfaces (Nielsen, 1997). 

3. Test Users 

The primary concer regarding test users is that they should be as representative 
as possible of the intended users of the system. Sometimes the exact individuals who will 
be using a system can be identified. Typically, this case is specific to a system being 
developed for an organization or department within an organization. Another case is a 
system targeted at a certain type of user, such as lawyers, secretaries or warehouse 
managers. In this case the users can be more or less homogeneous, but it is desirable to 
select users from as many different locations as possible. For software intended for the 


general population, almost anyone can serve as a test user. Lastly, care must also be 
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taken to ensure that both expert, novice, and mid skill level users are chosen (Neilsen, 


199%): 


4. Test Tasks 


Test tasks should be as representative as possible of the uses to which the system 


will eventually be fielded. They should also reasonably cover the most important parts of 


the user interface, be small enough to be completed within the test time limits, yet not 


become trivial. The first task should be extremely simple to guarantee the user an early 


success experience to boost morale. Conversely, the last task should provide the user 


with the feeling of accomplishment (Gaylin, 1986). 


Usability tests typically have four stages (Nielsen, 1997): 


I. 


Preparation: Ensure the test room is ready and that the computer system is 
in the start state, and that all test materials and instructions are available. 
Introduction: Welcome the test user and give a brief expalnation of the 
purpose of the test, to include computer setup if necessary. 

Running the test: During the test itself, the experimenter should normally 
refrain from interacting with the user. The exception to this rule is the 
case where the test users is clearly stuck and becoming quite unhappy with 
the situation. Caution must be exercised not to help the user too early. 
Debriefing: After the test, debrief the user and get any subjective input as 
soon as possible. During the debrief, extract from the user their input 
regarding the test itself. Also, as soon as possible, ensure that all results of 


the test have been collected and properly labeled. 


Consideration of the required tasks is critical to properly setting up the test. It is 


important that the test user feel as if their input is significant, beneficial and appreciated. 
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5. Performance Measurement 

Measurement studies for the basis of much traditional research on human factors. 
User performance is almost always measured by having a group of test users perform a 
predefined set of test tasks while collecting time and error data (Nielsen, 1997). There is 
the potential, however, to measure something that is poorly related to the property which 
is attempting to be assessed. Goals are generally abstract, so care must be taken to break 
them down into components with specific usability attributes, which are to be measured 
by specific tasks. Some usability studies use qualitative methods rather than exact 
measurements. The focus of the test could be to determine which aspects of the system 
work well and which are troublesome, yet not have any exact measures to differentiate 
the two. To mine all possible observations of qualitative data analysis requires extensive 
engineering experience. Even inexperienced usability engineers, however, can usually 
ascertain major qualitative findings, as they are more likely to be readily apparent 


(Nielsen, 1997). 


G. SUMMARY 

Naval Aviation has made significant strides in reducing the mishap rate in the last 
half century. These advances can be attributed to more standardized investigation and 
reporting systems, the use of systems engineering and the application of human error 
causation theory on mishap cause factors. Significant amounts of data are being 
collected, cataloged and stored for future reference. Computer database systems make 


the organization of this data relatively simple. For Naval Aviation to take the next step, 
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however, a robust system for analyzing stored data for trend analysis and prevention is 
needed. 

In an effort to provide a more useful data analysis system, HFACS was 
developed, then expanded to cover MRMs in HFACS-ME. Proven to be effective in 
capturing the nature of, and relationships among, latent conditions and active failures. 
The HFACS-ME taxonomy was developed into a laptop prototype database tool, 
MEIMS. In a limited usability study, MEIMS was discovered to have great potential. To 
fully attain its potential, however, MEIMS must be proven to be a user friendly system 
which fleet users will embrace. Further system development and more rigorous usability 
testing is required. With proper advances, MEIMS can be especially beneficial in 


reducing aviation maintenance related mishaps. 
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Il. METHODOLOGY 
A. RESEARCH APPROACH 

A desktop investigating, reporting and analysis system focusing on maintenance 
error in aviation mishaps can greatly advance efforts to enhance safety and reduce 
mishaps. This information management system’s database is populated with historical 
records of aviation maintenance mishap information. Additionally, a cursory training 
lesson and a hands-on experience tool are developed which assists the participants in the 
use of the database. Next, the prototype version of the tool is distributed to two sample 
groups; one getting the training lesson and an operator's manual and the others getting the 
training session, an operator's manual, and a hands-on experience session. 

Participants in both groups are then provided a prepared task list that required 
them to navigate through the system. At the completion of the task list, all participants 
have viewed all portions of the prototype system, and have had the opportunity to form 
an opinion on its effectiveness. The two groups’ performance is evaluated to determine 
the usefulness of the hands-on experience session in completing the assigned tasks. 
Additionally, all participants complete an exit survey composed of demographic 
background questions, opinion items of the prototype system (including improvements), 
opinion items on the value of the training lesson and the hands-on experience session. 
They also answer open-ended questions on the prototype system, training session and 


hands-on experience session. 
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B. DATA COLLECTION 

1. Participants 

Students attending the Naval Postgraduate School (Monterey, CA) serve as 
participants in the study. Most of the participants have recently come from fleet 
department head tours or embarked airwing staffs. These aviators are from all aircraft 
communities and within the U.S. Navy and Marine Corps. All have had some experience 
with mishap information reports and the data contained therein. They are representative 
of the type of people that will likely be the primary users of the tool. The participants 
were split into two groups of ten. The groups were distributed evenly matching rank, 
branch of service, type aircraft flown and years of aviation experience. One group 
receives a training lesson on the tool along with the instruction manual and the other 
group gets the training lesson, instruction manual and an additional in depth tutorial. 

2. Apparatus 

The participants are introduced to the Human Factors Analysis and Classification 
System-Maintenance Extension (HFACS-ME) taxonomy and to the Maintenance Error 
Information Management System (MEIMS) prototype data base tool. Additionally, the 
students have access to three computer labs at the School of Aviation Safety via login [ID 
and password to a group account. Each computer in the laboratories is a Pentium I, 
Windows 2000 operating system, with 15-inch monitor of 800 x 600 resolution or better. 
Each has a full functioning prototype of the tool loaded onto it. Prior to logging on, both 
groups are given a cursory training lesson and instruction manual in the use of MEIMS. 
Additionally, one group is provided with an additional in depth tutorial in HFACS-ME 


taxonomy and MEIMS capabilities. After a participant gains access to the computer, the 
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icon “MEIMS Tool” is found on the computer desktop. Once the icon is selected, the 
prototype opens using Microsoft Access 2000. 

3. Instrument 

The participant completes a usability test consisting of eleven tasks. After 
completion of the test, an exit survey is constructed for completion by the participants 
consisting of three parts: (1) demographics of the participant, (2) Likert-type questions 
assessing feelings towards the prototype tool (including improvements), the hands-on 
experience session and tutorial (as applicable), and (3) open-ended items to elicit 
subjective responses. Participants receiving the tutorial answer additional Likert-type 
questions and open-ended questions regarding the tutorial. Survey questions are designed 
to determine if the tool meets user investigation, reporting, and analysis requirements. 
The survey also queries the participants getting the tutorial in its value. Additionally, 
participant performance between the two groups is evaluated to determine the 
effectiveness of the tutorial. 

Collection of demographic information is accomplished through the participant 
selecting from a list of descriptors (squadron, experience with computers, software 
familiarity, and operating system knowledge). The Likert-type questions use a five point 
rating scale with verbal anchors: Strongly Agree, Agree, Neutral, Disagree, and Strongly 
Disagree. Open-ended questions are included to gain inputs on the overall impression of 
the training lesson, prototype tool, the hands-on experience session, the tutorial (as 


applicable), and recommendations for improvement. 
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4. Procedure 
MEIMS was revised and updated to include data derived from an available 
database; a sample group of users will measure its effectiveness. The participants are 
split into two similar groups. One group 1s provided an in depth tutorial (See Appendix 
B) prior to using MEIMS. The perceptions and performance of both groups are measured 
to determine the value of the MEIMS and the tutorial. 

All Participants are given an overview presentation of the study consisting of a 
projected computer demonstration of the prototype tool and distribution of materials 
necessary to carry out the user test. The overview consists of: 

e Instructions for Accessing the Prototype Tool - information to log on and open the 
prototype. 

e MEIMS Evaluation (Appendix C)--a series of planned navigation routes within 
the prototype whereby the participant would be able to view the entire system. 

e MEIMS Exit Survey (Appendix C)--Participants complete an exit survey 
composed of demographic background questions, impressions of MEIMS and the 
tutorial (as applicable) and open-ended question requesting participant’s opinions. 
All participants perform the above actions: access the prototype tool, navigate the 

system using the prototype task list, and provide feedback on the system by completing 
the exit survey. Additionally, the group receiving the tutorial makes comments on its 
effectiveness and usefulness. The performance of the tutorial group is compared with the 


non-tutorial group to determine the benefits of the tutorial. 
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C. DATA TABULATION 

The data is transcribed from the survey onto a Microsoft Excel 2000 spreadsheet. 
The Likert-type questions, based on a five-point scale, are coded into the software (1 
through 5) corresponding to the respective anchors (Strongly Disagree, Disagree, Neutral, 
Agree, and Strongly Agree). Specific categories (MEIMS is: Logical, Easy to Navigate, 
Very Interesting, Relevant and a Good Concept) are examined for significant differences 
between the tutorial and non-tutorial groups. Open-ended questions are tabulated by 


theme and number of responses. 


D. DATA ANALYSIS 

The data analysis tools of Microsoft Excel 2000 are used to generate descriptive 
statistics including the mean, standard deviation, and range of the collected data. The 
performance of the two groups is tabulated, measured and compared. Independent t-tests 
are conducted on the data to determine significant differences between the tutorial and 
non-tutorial groups. Participant responses to the open-ended questions are also explored 
to assess the perceived value of the hands-on experience session, the MEIMS tool itself 


and the tutorial. 
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IV. RESULTS 

A. TEST SAMPLE 

A usability test was administered to twenty students at the Naval Postgraduate 
School. They were randomly split into two groups of ten each. Many of the participants 
(n=17; 85%) were designated Naval Aviators or Naval Flight Officers and represented a 
cross section of the aviation commands that make up the squadrons in the Navy and 
Marine Corps. One group of ten received a formal tutorial prior to testing the MEIMS 
prototype, while the other did not. One of the non-aviators was in the tutorial group, two 
were not. Both groups were directed to access MEIMS and complete an eleven-item task 
list. Following the tasks, individual participants were asked to complete an exit survey. 
The survey consisted of demographic information, queries regarding their satisfaction 


with MEIMS and queries regarding their satisfaction with the tutorial, if applicable. 


B. TEST TASKS 

The tasks portion of the test was designed to not only introduce the participants to 
MEIMS and exercise some of its capabilities, but also to test the effectiveness of the 
tutorial. The tasks required the participant to access all functional areas of MEIMS. The 
tasks started relatively simply but became more complex as the participant became more 


familiar with MEIMS. Test performance is summarized in Table 2. 
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Table 2: Test Task Performance 


10 





The first task was accessing the program; all participants (n=20; 100%) were able 
to access MEIMS with no difficulty. The second task requested the participant’s opinion 
of the main menu. A number of participants from both the tutorial group (n=6; 60%) and 
the non-tutorial group (n=4; 40%) noted that the tab order was not sequencing properly in 
the main menu. The third task required the participant to query a type of aircraft and 
determine how many mishaps exist in the database for that type aircraft. Secondly, the 
participant had to identify and define the level 3 codes. All participants (n=20; 100%) 
were able to correctly complete this task. The fourth task asked the participant which of 
the seven categories was most useful and which was least useful. Aircraft model (n=17; 
85%) was considered the most useful, while Location of Mishap and Aircraft Type (n=6; 
30%) were each viewed as the least useful. 


The fifth task asked the participants to select their own criteria for a multiple 


query. Some participants (n=5; 25%) created queries that were too restrictive for the 
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limited number of mishaps in the prototype database. Once they viewed the error 
message that their query was too restrictive, they were able to reselect and complete the 
query. Both the tutorial (n=8; 80%) group and non-tutorial (n=7; 70%) group thought 
this feature was extremely useful. The sixth task required the participant to access the 
HFACS-ME Summary query and determine how many mishaps were in the database. 
All participants (n=20; 100%) correctly obtained this information. The participant then 
had to determine how many level one Worker Condition mishaps were in the database 
and how many level two Medical conditions were in the database. All of the tutorial 
group (n=10; 100%) correctly answered this question, however not everyone in the non- 
tutorial group (n=7; 70%) correctly answered it. The seventh task asked the participant to 
determine the number of mishaps of another type aircraft. All participants (n=20; 100%) 
correctly determined the number of mishaps. 

The eighth task required the participant to determine the number of Detached 
mishaps in the Mishap Distribution — Location Report. In the tutorial group, almost all 
(N=9; 90%) correctly answered this question. In the non-tutorial group, over half (n=6; 
60%) correctly answered it. The ninth task asked the participant to graph two type 
aircraft on the X-Axis versus HFACS-Level One codes on the Y-Axis. All participants 
(n=20; 100%) were able make the graph. Some participants in both the tutorial (n=4; 
40%) and the non-tutorial (n=2; 20%) groups noted that the graph provided frequency of 
mishaps, vice some normalized baseline, like percent of mishaps per type aircraft. The 
tenth task was to input two mishaps into the database. All members of the both groups 


(n=20; 100%) were able to successfully enter mishaps into the database. [t was noted by 
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many participants in both groups (n=16; 80%) that the database does not adequately error 
check the information being input. 

The eleventh task asked the participant to check the Chronological Listing of the 
mishaps that had been input into MEIMS. Again, all the participants (n=20; 100%) 
correctly completed this task. Overall the performance of the tutorial group was slightly 
better than the non-tutorial group (See Figure 7). In the tutorial group almost all (n=9; 
90%) had no errors, while in the non-tutorial group less than half (n=4, 40%) had no 
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Figure 7: Test Task Performance 
The mean percentage of correct responses for the tutorial group was 99.1 (SD= 
2.84, RNG = 10). whereas the non-tutorial group mean was 93.7 (SD = 6.07, RNG = 19). 
An independent tequai teSt Comparing mean tutorial and non-tutorial group performance 
was conducted. It assumed equal n, no difference between group means, and 
homogeneity of variance. The observed tequa Of 2.55 was significant at the p = 0.02 level 


for a two tailed test (See table Al, Appendix A). Given the variance for the tutorial 
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group was 8.1 and for the non-tutorial group was 36.9, a second t-test was run for 
unequal variance to reduce the chance of Type I error. The results are presented in Table 


3. The observed tynequai Of 2.55 was significant at the 0.02 level. 


Table 3: Task Performance Two-Sample t-Test Assuming Unequal Variance 


Non-utora 


n 10 10 
[Men | got]. 0 

Variance Lap 36.90 

df 13 

t Stat 2,90 


P two-tail 


C. DEMOGRAPHIC INFORMATION 

Part I of the exit survey requested demographic information from the participants. 
It established computer experience levels to determine if computer or aviation experience 
had an impact on that participant’s ability to utilize, or their satisfaction with, the MEIMS 


tool. Demographic information is summarized in Table 4. 
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Table 4: Demographic Information 


Number of Participants 
ee 


SQDRN MX 
2 + YRS Computer EXP 

















Microsoft Office 
Lotus Smart Suite 
Corel Office 


Word Processina 


Graphic Software 


pense | 


Windows (3.1-2000 
Windows NT 


Macintosh 


Presentations oe 





Linux 





Question one determined that 9 of the tutorial group participants had been 


members of aviation units that performed squadron level maintenance (n=9; 90%). The 


other member of the tutorial group was from a non-aviation background (n=1; 10%). In 


the non-tutorial group, 7 participants were members of aviation units that performed 


squadron level maintenance (n=7, 70%), one was a member of a staff that did not 


perform maintenance (n=1; 10%), and two were from non-aviation backgrounds (n=2; 


20). Question two indicated that all participants had at least two years of experience 


using a computer (n=20; 100%). Question three determined that all participants (n = 20; 
100%) were users of Microsoft Office. Minimal numbers in either group had used either 


Lotus Notes (n = 2; 20%) or Corel/Word Perfect (n = 1; 10%). Question four established 
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a participant’s familiarity with different software applications; all participants (n=20; 
100%) were familiar with processing, spreadsheet, presentation, and e-mail. In the 
tutorial group, four were familiar with database applications (n=4; 40%), while in the 
non-tutorial group, that number was 6 (n=6; 60%). Question five determined that all 
participants (n=20; 100%) were familiar with Windows 97/2000, Windows NT operating 
systems, or both. The MEIMS tool was loaded on computers running the Windows NT 


operating systems. 


D. USER SATISFACTION WITH MEIMS TOOL 

1. Impressions of MEIMS 

Part II of the exit survey requested the participant’s impressions of the MEIMS 
tool and its value to Naval Aviation. Participants responded to five statements using 
Likert type responses selecting from one of five responses: strongly agree, agree, neutral, 
disagree, and strongly disagree. Values of five through one respectively were assigned to 
the statements. Additionally, participants could make subjective comments on any of the 
statements. 

Statement one asked whether or not a participant found MEIMS to be presented in 
a logical form. The histogram of the frequency distribution for statement one is 
presented in Figure 8. The mean for the tutorial group was 4.4 (SD = 0.52; RNG = 2), 
whereas the non-tutorial group mean was 4.1 (SD = 0.57; RNG = 3). All participants in 
the tutorial group agreed that MEIMS presented the information in a logical fashion 


(n=10; 100%), while in the non-tutorial group it was less than unanimous (n=9; 90%). 
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Figure 8: MEIMS is in a Logical Form 


Statement two asked about the ease of navigation of the prototype. The histogram 
of the frequency distribution for statement two is presented in Figure 9. The mean for the 
tutorial group was 4.0 (SD = 0.5; RNG = 3), whereas the non-tutorial group mean was 
3.5 (SD = 0.97; RNG = 4). A majority of participants in both the tutorial group (n=9: 


90%) and the non-tutorial group (n=6: 60%) agreed that MEIMS was easy to navigate. 


# Responses 





Strongly Agree Neutral Disagree Strongly 
Agree Disagree 
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Figure 9: MEIMS is Easy to Navigate 
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Statement three asked if the participants felt MEIMS was “very interesting.” The 
histogram of the frequency distribution for statement three is presented in Figure 10. The 
mean for the tutorial group was 4.2 (SD = 0.63; RNG = 3), whereas the non-tutorial 
group mean was 3.7 (SD = 0.82; RNG = 4). Most participants in both the tutorial group 
(n=9; 90%) and the non-tutorial group (n=7; 70%) agreed that MEIMS was very 


interesting. 
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Figure 10: MEIMS is Very Interesting 


Statement four asked about the relevance of MEIMS to aviation maintenance 
operations. The histogram of the frequency distribution for statement four 1s presented in 
Figure 11. The mean for the tutorial group was 4.6 (SD = 0.52; RNG = 2), whereas the 
non-tutorial group mean was 4.5 (SD = 0.53; RNG = 2). All participants agreed that 


MEIMS was relevant (n=20; 100%). 
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Figure 11: MEIMS is Relevant to Maintenance Operations 

Statement five asked whether prototype concept was a good one. The histogram 
of the frequency distribution for statement five is presented in Figure 12. The mean for 
the tutorial group was 4.6 (SD = 0.52; RNG = 2), whereas the non-tutorial group mean 
was 4.5 (SD =0.53, RNG = 2). Again, all participants agreed that the concept of MEIMS 


was a good one (n=20; 100%). 
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Figure 12: MEIMS Concept is Good 
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Independent tequa tests comparing tutorial and non-tutorial responses to the Likert- 
type questions were conducted. They assumed equal n, no difference between group 
means, and homogeneity of variance (See Appendix A). For the Logical, Navigation, 
Relevance and Concept categories, no significant difference was noted. For the 
Interesting category, an observed tequai of 2.59 was significant at the p = 0.02 level for the 
two tailed test (See table A4, Appendix A). Given ne ocr for the tutorial group was 
0.28 and for the non-tutorial group was 0.68, a second t-test was run for unequal variance 
to reduce the chance of Type I error. The results are presented in Table 5. The observed 
tunequal Of 2.59 was significant at the 0.02 level. 


Table 5: MEIMS is Interesting Two-Sample t-Test Assuming 
Unequal Variance 


[wean | itso | 
variance | ozs | ge 
an 


0.68 


2. Open-ended Questions 


















Part III of the exit survey contained three open-ended questions regarding the 
participants overall satisfaction with MEIMS. All participants made constructive 


criticism, although overall, the response was very positive. The comments indicated that 
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MEIMS is currently a good tool that has the potential to be extremely valuable instrument 
in the prevention of mishaps (See Table 6) 

Question one asked the participant to list the most positive aspects of the 
prototype. Overall, the response was positive. Twelve participants commented to the 
effect that MEIMS would make an immediate, positive impact on safety in the fleet. 

Question two asked for the most negative aspects of the prototype. Most 
comments were problems or suggested improvement to the interface. Such items 
included error checking capability for data input, enlarge text boxes so the data can be 
completely read, line wrap the Contributing Factors text box so that the factors are not cut 
off after one line, and improving the tab function in the main menu, improve the page to 
page navigation. Numerous suggestions were made for “normalizing” the graphing 
functions. Several participants suggested a “find” mode in reports to prevent paging 
through them to find the data desired. Responses produced no discernable difference 
between the tutorial and non-tutorial groups regarding their impression of MEIMS. 

Familiarity and knowledge of HFACS-ME terminology, however, was a distinct 
problem for several of the non-tutorial group. Seven participants made some comment 
regarding HFACS-ME terminology. Most found it hard to understand. Conversely, none 
of the tutorial group, to whom the HFACS-ME terminology was thoroughly explained, 
had a negative comment regarding HFACS-ME terminology. 

Question three requested suggestions for changes to MEIMS. Most participants 
re-iterated their previously mentioned “negatives.” One participant made the suggestion 
that the database should cross-reference the MIR messages for each mishap. Seventeen 


made suggestions regarding navigation, error checking and GUI interface. One 


46 


participant mentioned making MEIMS available on the World Wide Web. There were no 
noteworthy differences between the tutorial and non-tutorial groups in response to this 
question. 


Table 6: Responses to Open Ended Questions 
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E. USER SATISFACTION WITH THE TUTORIAL 

1. Impressions of the Tutorial 

Part IV of the exit survey requested the participant’s impressions of the tutorial. 
Participants responded to five statements using Likert-type responses selecting from one 
of five responses: strongly agree, agree, neutral, disagree, and strongly disagree. Values 
of five through one respectively were assigned to the statements. Additionally, 


participants could make subjective comments on any of the statements (See Table 7) 
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Table 7: Summary of Responses to Tutorial 


} 
Tutorial | Strongly Agree Acree | Neutral : Disagree | Strongly Disagree 


| 
‘Beneficial 3 | 6 | 1 | 0 | 0 
Made Eval Easier | 6 | 3 0 1 | 0 
MmeestnG | 5 | = | 0 | 0 | 0 
Eee | 8 2 0 : 0 0 
Good Concept 6 | < 0 | 0 | 0 


Statement one asked whether the tutorial was beneficial. The histogram of the 
frequency distribution for statement one is presented in Figure 13. The mean for the 
tutorial group was 4.2 (SD = 0.63; RNG = 3). All but one participant in the tutonal group 
agreed that the tutorial was beneficial (n=9; 90%). The participant that was neutral 


commented that a user could figure MEIMS out in “15-30 minutes” without the benefit of 


the tutonial. 
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Figure 13: Tutorial is Beneficial 


Statement two asked if the tutorial made the usability test easier to complete. The 
histogram of the frequency distribution for statement two is presented in Figure 14. The 
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mean for the tutorial group was 4.54 (SD = 0.93; RNG =4). Almost all of the 
participants in the tutorial group (n=9; 90%) considered the tutorial to be beneficial. The 


participant that disagreed restated the reason given in the item above. 
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Figure 14: Tutorial Made Evaluation Easier 
Statement three asked if the participants felt the tutorial was interesting. The 
histogram of the frequency distribution for statement three is presented in Figure 15. The 
mean for the tutorial group was 4.5 (SD = 0.53; RNG = 2). All participants in the tutorial 


group found it to be interesting (n=10; 100%). 
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Figure 15: Tutorial is Interesting 
Statement four asked about the relevance of tutorial to MEIMS. The histogram of 
the frequency distribution for statement four is presented in Figure 16. The mean for the 
tutorial group was 4.8 (SD = 0.42: RNG = 2). All participants agreed that the tutorial 


was relevant to MEIMS (n=10; 100%). 
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Figure 16: Tutorial Relevant to MEIMS 
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Statement five asked whether the concept of a tutorial was a good one. The 
histogram of the frequency distribution for statement five is presented in Figure 17. . 
The mean for the tutorial group was 4.6 (SD = 0.52; RNG = 2). Again, all participants 


agreed that the tutorial concept was a good one (n=10; 100%). 
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Figure 17: Tutorial Concept is Good 

2. Open-ended Questions 

The last portion of the exit survey contained three open-ended questions regarding 
the participants overall satisfaction with the tutorial. The participants were asked about 
the positive and negative aspects of the tutorial. Lastly, the participants were encouraged 
to suggest changes to the tutorial. The comments made were overall positive. Some 
constructive criticism was made, aimed at making the tutorial better. 

Question one asked the participant to list the most positive aspects of the tutorial. 
Comments referred to the tutorial as “comprehensive,” “focused on the user,” and 
“providing the necessary background” to take the test. One participant stated that the 


tutorial was thorough without running too long. 
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Question two asked for the most negative aspects of the tutorial. There were 
fewer negative comments than positive ones. They included deleting the historical 
references in the beginning of the tutorial, making the slides “less busy,” and too much 
redundancy of the HEACS-ME codes. 

Question three requested suggestions for changes to the tutorial. There was only 
one comment made regarding changes to the tutorial (n=1; 10%). It was suggested that 
the tutorial should be self-study, providing the participant either the slides (with notes), or 


on a desktop from which to view the tutorial. 


V. SUMMARY, CONCLUSIONS AND RECOMMENDATIONS 


A. SUMMARY 

Naval Aviation is in the unenviable position of increased demand for high levels 
of operational readiness, equipment availability, and personnel training within a climate 
of reduced budgets. The notion of doing more with less is an everyday reality. To meet 
this demand, asset preservation is vitally important, as funding for replacements does not 
appear likely in the near future. In this fiscally strained environment, the costs of Naval 
Aviation mishaps, in terms of mission capability, operational readiness, and material 
status, are too high. Reducing the number and rate of mishaps occurrence is paramount 
to the success of Naval Aviation. Furthermore, reducing “avoidable” mishaps, those 
involving human error, is an absolute must. Human error in aviation mishaps must be 
identified, and appropriate intervention strategies developed, aimed at reducing the 
causes of those errors. Conventional accident causation theories suggest that accidents 
are the result of a complex combination of errors. Naval Aviation must target 


intervention strategies in key areas to effectively combat human error. 


Initial efforts were made solely in the area of aircrew error reduction, and a robust 
classification taxonomy, Human Factors Analysis and Classification System (HFACS) 
was developed. It was soon realized that to meet the goals in mishap reduction, Naval 
Aviation had to look beyond aircrew error and study maintenance human error. To that 
end a Maintenance Extension, HFACS-ME was built. This expanded taxonomy proved 
to be an acceptable method for classifying human factors in maintenance mishaps. Using 


HFACS-ME to study mishap causes, a clear picture of the human factors contributing to 


mishaps can be drawn. 
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To fully utilize the HEFACS-ME taxonomy, a prototype database tool, 
Maintenance Error Information Management System (MEIMS) was created. It enables 
better access to the data, query, graphing and reporting functionality, and the ability to 
add new information to the database. In an initial usability study, MEIMS was 
determined to be a valuable tool in the effort to reduce human error in maintenance. A 
MEIMS revision was completed, updating and improving it, and targeting many of the 
issues detailed in the initial usability test. Additionally, a users tutorial was built to assist 
the targeted user in the introduction to MEIMS. This study investigated the validity of 
the MEIMS tool and the tutorial. MEIMS is developing into a robust tool not only for the 
analysis of mishap causal factors, but also an effective weapon in the development of 
prevention programs tailored to human error in maintenance mishaps. Its positive impact 


on the fleet will be immediate and far-reaching. 


B. CONCLUSIONS 


The participants of the study were overwhelmingly supportive of MEIMS and the 
tutorial. Results from the usability test and the Likert-type responses indicate that the 
tutorial was beneficial and made the test easier. Participants in the tutorial group 
performed better in the tasks portion than the non-tutorial group. Based upon the 
independent t-tests conducted, there was a significant performance difference between 
the two groups, with the tutorial group performing better. The response to the Likert-type 
questions did not indicate a significant difference between the tutonal and non-tutonal 
groups (as determined by independent t-test) regarding MEIMS being logical, easy to 


navigate, relevant to maintenance operations and a good concept. Regarding MEIMS 
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being interesting, an independent t-test indicated a significant difference between the 
tutorial and non-tutorial groups, with the tutorial group finding MEIMS more interesting. 
This data indicates an increasing interest level with increased exposure and knowledge of 
MEIMS. The tutorial group’s support for MIEMS was more enthusiastic. Although both 
groups strongly advocated MEIMS relevance to maintenance operations and endorsed it 
as a good concept, the tutorial group was more positive in supporting jt. The participants 
of the tutonal provided excellent feedback and also solidly endorsed it as a 
ociecier link with MEIMS. 

All participants made valuable comments and suggestions. Numerous previously 
unrecognized GUI deficiencies were identified. Additionally, data error checking, 
graphing and reporting anomalies were detailed. Most beneficial, however, were the 
suggestions to make MEIMS a better product for the fleet. Specifically linking date- 
time-groups of mishap investigation reports to MEIMS was an especially astute 
suggestion. Also, suggesting MEIMS for the World Wide Web, a pre-planned 
progression for future iterations, was certainly perceptive. 

The potential of MEIMS is expansive. MEIMS should be expanded to include 
other services, allies and NATO mishap information to broaden the scope from which 
data is drawn. Efforts are ongoing to make a variation of MEIMS that would include 
civilian and commercial aviation. That would enable military users an even larger pool 
of information from which to draw data and will be helpful in providing insights and 
parallels between military and civilian/commercial aviation safety. MEIMS should soon 


be available on the World Wide Web. It will be an excellent tool for users, from Naval 


ao 


Aviation’s highest executive levels down to squadron work centers, to derive information 


and to provide the necessary tools to prevent maintenance related mishaps. 


C. RECOMMENDATIONS 


For any software tool to be effective, it must be considered easy to use and 


effective. That notion is especially critical in a too] which the fleet 1s expected to use. If 


it is not easy to use and effective, it will remain on the shelf. To that end, the following 


recommendations are made for the MEIMS tool: 


Include the tutorial as a part of MEIMS, upgrading as necessary 

Provide better navigation from MEIMS functions to the main menu 
Improve GUI anomalies including the tab function on the main menu, 
improving text boxes so that all data can be read by the user 

Improve the query causal factors to add text wrap, so mishap information 
will not be cut off. 

Provide better error-checking during data input 

Add a find mode in the reports function to enable the user to quickly 
access the desired data without paging through the report. 

Normalize graph presentations to provide a “weighted” view of data 
Provide more sub-menus to expedite multi-layer functionality 

Increase the volume of the database to include more mishaps 

Provide a link to the date-time-group of the Mishap Investigation Report 


Make MEIMS and the tutorial available over the World Wide Web 
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MEIMS is already a robust tool for analyzing and reporting data regarding 
maintenance human error. Making the above improvements will make it an even better 
tool. Once it 1s available on-line with the proper training, it should be a well-used tool in 


the effort to preserve lives, material and readiness. 
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APPENDIX A 


T-TEST RESULTS FOR MEIMS EVALUATION AND PART II OF EXIT 


SURVEY 


1. MEIMS TASK PERFORMANCE 


The following is the complete t-test results from the MEIMS evaluation assuming 


equal n and variance. 


Table Al: Test Task Performance Two-Sample t-Test 


Tutorial 
| n 10 
: Mean | 99.10 
| Variance 8.10 
| Pooled Variance 22250 
| df 18 
| t Crit | 2.10 
| t Stat | 255 
| P two-tail | 0.02 


2. LIKERT-TYPE QUESTION RESPONSE 


Non-tutoria! 


= 
O 


93.70 
36.90 


The following are the complete t-test results from each of the Likert-type 


questions from Part II of the Exit Survey. The t-tests assume equal n and variance. 


Table A2: MEIMS is in a Logical Form Two-Sample t-Test 


| Tutorial | Non-tutoria! | 
| n 10 10 | 


410 


| Mean | 4.40 
| Variance | 0.27 0.32 
Pooled Variance | 0.29 
| Of | | 
| t Crit | 2.10 | 
t Stat | 1.24 
| P two-tail | 0.23 | 


Table A3: MEIMS is Easy to Navigate Two-Sample t-Test 









Tutorial Non-Tutoria! 





: : : 
Lon 10 | 10 | 


| 

Mean | 4.00 3.50 | 
| 
| Variance 0.22 0.94 
| Pooled Variance : 0.58 





Table A4: MEIMS is Very Interesting Two-Sample t-Test 


Tutorial Non-tutorial 


P two-tail 0.02 
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Table A3: MEIMS is Relevant to Maintenance Operations Two-Sample 


t-Test 
| Tutoria! | Non-tutorial 
n | 10 | 10 
Pooled Variance 0.27 | 
? 

Lor 

t Stat 0.43 

P two-tail 0.67 


Table A6: METMS Concept is Good Two-Sample t-Test 


t 
i 


Tutorial | | 
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APPENDIX B 


MEIMS TUTORIAL PRESENTATION WITH NOTES 
Slide 1] 









Maintenance Error Information 
a see (MEIMS) 


Human Factors Accident 
Classification System- 
Maintenance Extension 


LCDR Mike “Cracker” McCracken 
NPS ITM Curriculam 


F-14 RIO 





(Ensure that the students have copies of these presentation slides to use as a 
reference during the presentation) 


Welcome to the Maintnenancee Error Information Management System (MEIMS) 
brief. This prototype tool is built upon the Navy’s HFACS-ME (Human Factors 
Accident Classification System- Maintenance Extension) taxonomy. This 
presentation will detail the HFACS-ME model in detail, explain terminology and 
provide details. Please pay close attention and ask questions as necessary. After 
explaining the HFACS-ME taxonomy, the MEIMS tool, built upon that taxonomy, 
will be briefed and demonstrated. 
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Naval Aviation Flight Mishap Rate 
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Naval aviation has become considerably safer over the last few decades, but there is 
still a need for improvement. 


This chart shows the significant reductions in Naval Aviation mishap rates (per 100,000 
flight hours) since 1950. 


Improvements in Naval Aviation safety have primarily focused on engineering and 
administrative controls. They have included (but have not been limited to) design 
improvements (angled carrier decks), the creation of the Naval Safety Center, improved 
maintenance programs (NAMP), training (FRS, NATOPS), local safety programs, and 
system safety designated aircraft. 


The Mishap rate has actually been cut in half each decade since 1950, HOWEVER, there 
are still many Class A mishaps each year (155 total Class A’s for FY 95-99) which result 
in unacceptable loss of life, disabling injuries, and hundreds of millions of dollars in 
damages. So. again, improvements are necessary. 


FY 95-99 Total Navy/Marine Aviation Class A Mishaps 
Flight Hours _ Flight Mishaps_ Rate 


mus 1,656,450 34 20S 
FY96 1,650,026 36 2.18 
FY97 176282507 af i) fae 
FY98 1018, 963 36 Cac 
FY99 Vo2/ oe Ze 1.44 
Fes 7.875.278 ers ie 
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Engineering & Administrative Controls 
have Impacted Hardware Reliability, but....: 


| 
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22 Class A FMs in 1999 


17 Had Human Error 
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Engineering and administrative controls have significantly impacted hardware reliability (the 
aircraft/machinery are more reliable), but mishaps still occur. Why? 


This chart shows that over the last three decades. the material factors that cause mishaps (e.g., machine 
reliability) have steadily decreased. Human error has also decreased (with training. proper supervision, 
etc.), but at a slower rate so that it has resulted in a higher percentage of the factors that cause mishaps. In 
fact, the percentage of human errors in mishaps is now increasing. 


In 1999. there were 22 Class A Flight Mishaps; 17 had human error involvement (77%). 
Again, there were 155 Navy/Marine Class A’s for FYs 95-99! 
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Mishap Data Analysis Focus: 
Naval Aviation Class A FM Causal Factors 





Initial emphasis on “Pilot Errer’; 
_.oeet?? butte reach goal of cutting human 
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NOTE: Miushaps are caused bv more than one factor. therefore. the chart percentages total more 
than 100%) 





To further reduce the mishap rate, we must identify and correct the causes of mishaps. 


Human error is involved in the majority of the determined causal factors in this chart, with material failures 
listed in 39% of the mishaps. Human error preventative measures have primarily been concentrated on 
aircrew and supervisory issues (e.g., training, SOPs). However, to reach the Human Factors Quality 
Management Board’s goal of reducing mishaps by 50%, we must also focus on maintenance errors. 
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The Heinrich Ratio 





We have just reviewed some statistics on major mishaps; however, MAJOR ACCIDENTS ARE 
ONLY THE TIP OF THE ICEBURG. 


To make significant improvements in aviation maintenance error, we must thoroughly analyze the major 
mishaps. Fortunately, major accidents (Class A mishaps) are relatively rare events. However, this low 
mishap rate poses a new problem. There Is simply not enough information available from Class A mishaps 
to conduct an effective trend analysis. 


Solution: For every Class A mishap, there are numerous Class B, Class C, and hazard reports which have 
similar cause factors. We must, therefore, thoroughly investigate the “minor accidents” as well as the 
“majors” to discover these causes. 


Heinrich Ratio 

The relatively few number of catastrophic accidents are the “tip of the iceberg”. For every major mishap. 
there are 10 less serious accidents, 30 incidents, and 600 hazardous acts. The circumstances (cause factors) 
which raise the severity of the accident are identifiable in all levels of accidents through adequate 
investigation. 


Source: 

Heinrich, H.W., Petersen. D. & Roos. N. (1980). Industrial Accident Prevention: A Safety Management 
Approach (5th ed.) New York: McGraw-Hill. Retrieved from http://hfsky way.faa.gov/training_toc.htm 
(MRM training site). 
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The Costs of Navy 
Maintenance Error Mishaps 


(cost per year in 1997 dollars) 


Class A Mishaps $85.0 Million 


Class B Mishaps $3.5 Million 


Class C Mishaps $3.8 Million 





The Costs of Navv Maintenance Error Mishaps. 


The costs are calculated from the FY90-97 Maintenance Related Mishaps and adjusted to 1997 


dollars. 


NOTE: This only includes component replacement costs and man hours to restore the aircraft in operating 


condition. 


Class A 
Class B 
Class C 


It does not include personal injury, loss of life. property damages or liability. 


$85.0 million/vear 
$3.5 million/year 
$3.8 million/year 


Source: Aircraft Maintenance Human Factors PAT report to the Safety QMB. 27 June 1998. 
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OK, so we must investigate all mishaps. Are there better or different methods of investigation? Has 
anything been overlooked? 

Occasionally. an accident is so unique and overwhelming that it sends shock waves throughout the 
industry. These “watershed events” provide a change in focus and are major catalysts for change. One of 
the more famous “‘watershed events” was the 1996 Nashville, Tennessee F-14 mishap. This mishap had no 
material factors...it was a human error mishap. 


Following this mishap, the Commander, Naval Air Forces Pacific (COMNAVAIRPAC) empanelled a 
Human Factors Quality Management Board (HFQMB) to identify human factors threats and develop 
intervention methods to combat them. 
The HFQMB determined that Human Factors was the key to reducing the mishap rate to 1.0 (the 
desired 50% reduction per decade with current rates stagnate near 2.0) 
The HFQMB adopted a three-prong approach to study human errors in mishaps: 
Mishap data analysis to identify hazards and risks 
Benchmarking (of other military services, NATO partners, private industry, etc.) to uncover 

best practices and find process improvements 
Developing a Command Safety Assessment 
The QMB’s initial efforts were primarily directed at aircrew errors, but the QMB later directed a 
Human Factors in Maintenance and Material (HFAMM) Process Action Team (PAT) to address 
maintenance errors, as well. 
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: HFACS- Maintenance Extension | 


Management Conditions 


Working Conditions 


Maintainer Conditions. 
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HFACS was developed to identify factors that relate to aircrew error. The Maintenance Extension 
(HFACS-ME) was added to analyze human error in maintenance related mishaps. So how does 
HFACS identify the factors specifically related to aviation maintenance accidents? 
The “Maintenance Extension” taxonomy for HFACS was developed to further classify causal factors that 
contribute to maintenance related mishaps. This addition to HFACS consists of four broad human error 
categories: 

Management Conditions (latent) 

Working Conditions (latent) 

Maintainer Conditions (latent) 

Maintainer Acts (active) 
But does it work? YES! HFACS has been applied effectively to the analysis of previous Navy mishaps. 
Additional causal factors were identified, and more importantly, the classification or coding of these factors 
was incorporated into an already extensive database of accidents and incidents. Standardized coding allows 
idenufication of trends throughout the various accidents and will enable analysts to offer further prevention 
strategies. 
Most important: This DOES NOT require a reinvestigation of the mishap. It can be accomplished, 
with minimal training, by investigators or analysts simply coding the narratives and causal factors that 
previously were reported. 
Due to its success: 
- HFACS has been adopted for inclusion into the Naval Aviation Safety Program ( OPNAV 3750.6R) 
- It has influenced policy changes to the Navy's Maintenance Program (4790 series) 
- And is being used to further the effectiveness of Naval Aviation’s methods of accident investigation, 
reporting, and database entry. 
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The HFACS-ME taxonomy also includes three orders of error: first, second, and third that reflect a 


macro to micro perspective. 


The levels provide sufficient description to develop a searchable database and are useful for distinguishing 
supervisory areas of action. For example, a First Order category could be briefed at the Executive level, a 
Second Order at the Vice President/Senior Manager levels. and the Third Order can be corrected by Safety 


Officers and front line supervisors. 


An Example: Maintainer Conditions (First Order), Readiness (Second Order), and 


Qualification/Certification (Third Order) 


Wd 
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HFACS - Maintenance Extension Model | 


ORGANIZATIONAL CLIMATE 


Supervisory Bt beter. b ber bag Working 
Conditions Conditions Conditions 


Maintenance 
Conditions 
Aircrew 
Actions 





This model provides a realistic example of the interaction between the Error Categories. 


Supervisory Conditions, Maintainer Conditions, and Working Conditions may 
INDEPENDENTLY or COLLECTIVELY affect the Maintainers Actions. 
The Unsafe Maintainer Act which follows may either: 

(1) Lead directly to a mishap or injury (Example: a maintainer runs a forklift into 
the side of an aircraft and damages it), or 

(2) Become a latent Maintenance Condition, which the aircrew would have to 
deal with on take-off, in-flight, or on landing. (Example: an improperly ngged landing 
gear that collapses on touchdown or an over-torqued hydraulic line that fails in flight and 
causes a fire) 


[tis important to note that Supervisory Conditions related to design for maintainability, 
prescribed maintenance procedures, and standard maintenance operations could be 
inadequate and also lead directly to a Maintenance Condition. 


Please note that this model utilizes only the First Order Error Categories (again, 
shown in red). So how does the Second Order Error Categories affect this 
relationship? (quickly go to next slide) 
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'HFACS-ME Framework 


Error Categories of HFACS Framework | 
FirstOrder | Secon Order 


Organizational | - Hazardous Operations « Inadequate Design - madequste 
Decurne mation | : |_e 
| - Inadequate Resources ~ adequate Processes 
i ; : , ie 4 
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5 inadequate Supervision =. - Inappropaate Operations ~~ Uncomecied Problem 
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Equipment ers 
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Violation 


The Second Order of Error Categories (shown in Blue) is simply a further breakdown, or description, of the 
First Order Error Category (so additional relationship models are not necessary). If we look back a couple 
of slides...(quickly go to next slide)... 
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HFACS- Maintenance Extension 
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...the Incorporation of the Second Order Error Categories would look like this. 


Supervisory Conditions are established at the Organizational levels (outside your command) or at your 
Squadron level. 

Working Conditions that affect one or more maintainers include the Environment, Equipment, and 
Workspace limitations. 

Maintainer Conditions directly impact an individual maintainer and may include Medical, Crew 
Coordination, or Readiness factors. 

Maintainer Acts can either be Errors or Violations. 


(NOTE: Multiple Error Categories may be factors in mishaps. For example, both Organizational 
and Squadron practices may combine to form Supervisory Conditions that lead to mishaps, or a 
combination of Equipment and Workspace conditions may promote an unsafe Working Condition.) 


Now, as vou probably surmised, the Third Order Error Categories are simplv a further breakdown 
of the Second Order Error Categories shown on this slide. 
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HFACS-ME Framework 


Error Categories of HFACS Framework 


Second Order Third Order 


Supervisory Organizational - Hazardous Operations - Inadequate Design - madeguare Documentation 
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Conditions Hazards 


Equipment 
- Damaged ~ Unavailable - Dated/Uncertiied 
Workspace 
- Confining ~ Obstructec - inaccessible 
Mainamer ] Medical - Mental State - Physical State - Limitation 
Conditions a. ; 
Crew Coordination - Communication - Assertiveness 
Adaptability/F lexibilty 
Readiness ean ae 


~ Training/Preparation - Certiication/Qualtication - Ininngement- 


Maintainer Acts | SOE |. Attention - Memory - Knewledge/Rule 
Based : : 

- Skil Based - Juégment/Decision-Making 
Violation ‘ie, es 7 x 





Here again, are the Third Order Error Categories (now displayed in Green). 


The most effective way to define and display the relationship of First, Second, and Third error 
categories is to examine each First Order Category independently. (quickly go to next slide) 


is 


Slide 14 


HFACS- Maintenance Extension 












Maintainer Conditions 


Fe ix Pid ee A « 
; mes sosainctton k 
| a ey é aA COPED! “i i, ‘4 5 €/3 te $ 





ISHAP/INJOR 






We will begin by examining the Supervisory Conditions First Order. (quickly go to next slide) 
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Supervisory Conditions 


Failed to Provide Guidance gam — ES Purchasing Failure 

Poor Risk Management he a =~ - ae ss | Deficiency not Corrected 
Operational Doctrine Errors i — Modified Equipment 
Failed to Provide Training 3 2 


J nadequate Resources| Inadequate Processes 


No/Poor Documentation Improper Manning Fail to Enforce Regulations 
Documents not Updated Lack of Funding Failure to Provide Oversight 
Documents unusable (CDs) Lack of Parts/Equipment Failed to Track Performance 





Under Supervisory Conditions, we have both Organizational and Squadron Second 
Order Error Categories. 
We will start with the Organizational Category. 


(Facilitator note: Read each term and its example below and then mention some of 
the additional examples listed in each box on the slide. Encourage discussion from 
students) 


Under the Organizational Second Order. we have the following Third Order Error 
Categories: 

Hazardous Operations An engine that falls off of a stand during a change out evolution 
due to an unforeseen hazard of a high seas state 

Inadequate Design The poor layout of system components that do not permit direct 
observation of maintenance being performed 

Inadequate Documentation A manual omits a step in a maintenance procedure, such 
as leaving out an o-ring that causes a fuel leak 

Inadequate Resources Insufficient funding, manpower, tools, parts and equipment to 
perform maintenance effectively and safely 

Inadequate Processes Limitations in teamwork, communication, and directives within 
and between organizations 








NOTE: The descriptions given. and the examples shown on the slide. are just that . 


.. examples. The purpose of this presentation is to establish a knowledge and 


understanding of the HEACS-ME Categories, not restrict their use to only a handful 
of scenarios.) 
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Inadequate Supervision 


Fail to Provide Guidance 
Fail to Provide Oversight 
Fail to Provide Training 
Fail to Track Performance 
Fail to Track Qualifications 
Failure to Inspect 


nappropriate Operations 


Uncorrected Problem 


Documents not updated 
No Corrective Actions 
Unsafe Condition not Reported 


Insufficient Resources 
Inadequate Bnief Times 
Improper Manning 
Inadequate Schedule 





Third 
Orders 


tienen et | 
Supervisory Misconduct 


Ignoring Risks 
Fail to Enforce Rules/SOPs 
Use of Unsafe Equipment 
Use of Untrained Personne! 


And under the Squadron Second Order Supervisory Condition, we have these Third 


Order Error Categories: 


Inadequate Supervision A supervisor who does not ensure that maintenance personnel 


are wearing required personal protective gear 


Inappropriate Operations A supervisor who directs a maintainer to perform a task 
without considering risks, such as driving a truck through a hangar 
Uncorrected Problem A supervisor who neglects to correct maintainers who routinely 


bend the rules when they perform a common task 


Supervisory Misconduct A supervisor who willfully orders a maintainer to wash an 


aircraft without proper safety gear 
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HFACS- Maintenance Extension | 
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Let us now examine the Working Conditions First Order Error Category breakdown. 
(Again advise students to follow along with their copy of the HFACS-ME Framework slide). 
(quickly go to next slide) 
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W eather/Exposure Environmental Hazards 



















Night visibility Extreme Temperatures 









: Slips/Fall 
W orkspace Illumination Ice on Equipment ain Ait 
Inadequate flashlights Visibility in Rain/Snow/Fog 
e Sunstroke 





Equipment/Manning Changes 
Inadequate Clothing 






Working Conditions have Second Order Error Categories of Environment, 
Equipment, and Workspace factors. 


The Environment Second Order can be further broken down into the following 
Third Orders: 

Lighting/Light A maintainer who is working at night on the flight line does not see a 
tool he left behind 

Weather/Exposure A maintainer who is securing an aircraft in a driving rain fails to 
properly attach the chains 

Environmental Hazards A maintainer who is working on a pitching deck falls from the 
aircraft. 


(Note: Environmental Factors are often not adequately reported.) 
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SMibtae. 
Orders 


Dam aged Unavailabiec Dated/Uncertified 


Limited Use Equipment used elsewhere Calibration Expired 

Unreliable Equipment not in inventory Open Purchase/U ncertified 
Gauge/Calibration Errors Equipment unusable ‘Extended Beyond Service Life 
Unsafe (brakes, electrical) : 





Equipment Third Orders are: 

Damaged A maintainer who is using a defective test set does not pre-check it before 
troubleshooting 

Unavailable A maintainer who starts working on landing gear without a jack because all 
are being used 

Dated/Uncertified A maintainer who uses an old manual because a CD-ROM reader is 
not available. 


(Note: Tools and Equipment Factors are also not adequately reported.) 
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Third. 
Orders 
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Insufficient Workspace Vision Obscured (fog.smoke) Inadequate Aircraft Design 


| Insufficient Maneuverability Vision Blocked (obstacles) fiinadequate Support Equipment 
Maintenance Hindered 





And finally, Workspace factors can be separated into these Third Order Error 
Categories: 

Confining A maintainer who is working in a hangar bay cannot properly position the 
maintenance stand 

Obstructed A maintainer who is spotting an aircraft with his view obscured by catapult 
steam 

Inaccessible A maintainer who is unable to perform a corrosion inspection that is 
beyond his reach. 


(Note: Workspace factors are also not adequately reported.) 
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Maintainer Conditions, the conditions that uniquely affect individual maintainers, are separated into 
Medical, Crew Coordination and Readiness Categories. (quickly go to next slide) 
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Mental State Phvsical State 
Complacency 


Distracted Insufficient Reaction Time 
Mental Fatigue Visual Limitation 


Life Stress Medical Tilness Incompatible Intelligence 


Misplaced Motivation Raysicart aupee Incompatible Aptitude 
Task Saturation Physical Capability 
Loss of Situational Awareness Physical Reach/Size 
Channelized Attention 





The Medical Third Orders are based on an individuals mental and physical abilities 
(and limitations) to perform a task at a certain time. 


They include: 

Mental State A maintainer who has a marital problem and cannot focus on a 
maintenance action 

Physical State A maintainer who worked for 20 hours straight and suffers from fatigue 
Limitation (Physical) A maintainer who is short cannot visually inspect an aircraft 
before it is launched 


(Note: Medical Factors are not reported adequately) 
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Maintainer Conditions 
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Terms not Standardized 
Hand Signals not Standardized 
Documentation Failures 
Documentation Delays 
Equipment Failures (radios) 
Equipment use (lights/whistles 
Inadequate Brief 


Fail to Brief/Make Suggestions 
Fail to Correct Discrepancies 
Fail to Confirm Messages 
Inattention to Feedback 
Waiver when Confronted 


Crew Coordination factors include problems in: 





Third 
Orders 


} Adaptabilitv/F lexibility | 


Emergency Response 
System Failure Response 
Changes to Routine 

Team member changes 


Communication A maintainer who leads a taxiing aircraft into another due to improper 


hand signals. 


Assertiveness A maintainer who performs a task, not in accordance with standard 
procedures, because the maintainer was overly submissive to a superior. 
Adaptability/Flexibility A maintainer who downplays a downing discrepancy to meet 


the flight schedule. 
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AMibtysl 
/}Orders 


| Training/P reparation | Certification/Oualification 


Not Trained for Task Not Certified in Task Intoxicated at Work 
Not Currentin Training Not Certified in Model Hung over 
Unrealistic Training Qualifications Expired Drug use 
Ineffective OJT 





And the last Maintainer Conditions category to discuss is Readiness. Its Third 
Orders involve: 

Training/Preparation A maintainer who 1s working on an aircraft skipped the requisite 
OJT evolution 

Certification/Qualification A maintainer who engages in a procedure that he or she has 
not been qualified to perform 

Infringement A maintainer who is intoxicated on the job. 


(Note: Readiness Factors are also not reported adequately.) 
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HFACS- Maintenance Extension 
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WISHAP/INJUB’ 


The final category is Maintainer Acts, which include Errors (mistakes) and Violations (willful acts). 
(quickly go to next slide) 
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Maintainer Acts 


Attention 


Missed Communication 
Loss of Situational Awareness 
Distracted 
Fail to Secure 
Left Unattended 
Improper Lookout 


| Knowledge/Rule based | Skill Based 


Delayed Response 


Insufficiently Trained Overuse of Controls 


Insufficient References 


Omitted Checkhist Item 
Incorrect Procedures 


Poor Technique 
Improper Cross-Check 


Omitted Step in Procedure 





Third 
Orders 


Procedure Mistakes 
Missed Task 
Sequence Errors 


Judement/Decision-M aking 


Exceeded Ability 
Poor Decision 
Wrong Response 
Misdiagnosed Problem 
Improper Procedure 


Maintainer Errors are of the following Third Order types: 
Attention A maintainer who misses a hand signal and backs a forklift into an aircraft 
Memory A maintainer who 1s very familiar with a procedure may reverse steps in a 


sequence 


Knowledge/Rule Based A maintainer who inflates an aircraft tire to a pressure required 


by a different aircraft 


Skill Based A maintainer who roughly handles a delicate engine valve will cause undue 


damage 


Judgment/Decision-Making A maintainer who fails to make appropriate decisions due 
to limited information, inadequate preparation, or perceived pressure to perform 
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Maintainer Acts | Third 
CERO Viclatton | Orders 


Infraction 


Did Not Follow Brief 
Bending of Regulations/SOPs 
Use of Incorrect Equipment 
Violated Training Rules 
Do Not Utthize Checklists 


Single Event to Save Time 
Violation to Expedite Mission| 
Use of Incorrect Equipment 

. Skip Publication Crosscheck 


Unauthorized Procedure —— ae. ee Deliberate destruction 
Not Curreny/Qualified | Attempt to injure/kill 
Exceed Equipment Limits Purposeful Mission Disrupuon 


Thrill Seeking 





There are also several types of Maintainer Violations. Their Third Order category 
is based upon frequency, intent, and supervisory involvement. 
Routine A maintainer who engages in practices, condoned by management, that bend 


the rules 
Infraction A maintainer who strays from accepted procedures to save time, bending a 


rule 
Flagrant A maintainer who willfully breaks standing rules disregarding the 


consequences 
Sabotage A maintainer who performs a criminal act to destroy equipment or endanger 


the lives of others 
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(FY 90-97) Human Error in | 
Maintenance Related Mishaps| 


470 Maintenance Related 
Mishaps Were Analyzed For 
Human Errors 


Classification Process 
Performed by Naval 
Maintenance Personnel 





HFACS, as previously stated. has been used effectively within the Navy. Without reopening an 
investigation, HFACS was applied to exisung Mishap Invesugation Reports. 
(go to next slide for genera! findings). 
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General Findings 


> Poor/non-existent maintenance 
procedures 


> Inadequate/poor supervision of 
maintenance evolutions 


> Miscommunication - supervisor 
to subordinate, pass-down, or 
shift turnover 


> Not using, lack of, or outdated 
publications 


> Violations - not following policy, 
procedures, checklists, etc. 





This slide shows the general findings of the analysis. I’m sure we are all familiar with these latent 
conditions. Let us now look at another study of 63 Class A mishaps between FYs 90-97. (go to next slide) 
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HFACS-ME Results of MRMs 
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First Order Analysis of 63 Maintenance Related Mishaps 





HFACS ANALYSIS OF Class a FYs 90-97 Maintenance Related Mishaps (MRMs). 

During FYs 90-97 there was a total of 63 Class A Mishaps. of which 61 were Flight, 0 were Flight Related, and 2 were 
Aircraft Ground. Two Navy Maintenance Officers and two Navy Chiefs used the HFACS Maintenance Extension to 
classify the human factors causes reported in these mishaps. They discovered the following profile of human errors: 
Management Conditions - 67% of all Naval Aviation Class A MRMs reported Squadron Supervisory 
Conditions, whereas 21% had Unforeseen Supervisory Conditions (not shown). 

Maintainer Conditions - 21% of all Naval Class A MRMs reported Medical, CRM, or Readiness Maintainer 
Conditions. Note: Maintainer Conditions were under reported, more are likely present and have an effect. 

Working Conditions - 3% of all Naval Class A MRMs reported Environment, Equipment, or Workspace 
Working Conditions. Note: Workspace Conditions were under reported, more are likely present and have an effect. 
Maintainer Acts - 75% of all Naval Aviation Class A MRMs reported Maintainer Errors, whereas 40% had 
Maintainer Violations. 

Clearly, latent conditions in the form of Supervisory, Maintainer, and Workspace factors are present that can impact 
maintainers 1n the performance of their jobs. However. many Maintaincr and Workspace Conditions are not reported 
due to the reporting system in place, perceptions of accident causation. or culture/climate issues. Specifically, 
inadequate supervision of maintenance evolutions, not ensuring personnel are trained and/or qualified, not enforcing 
rules, and poor communication characterize the majority of latent Supervisory Conditions. Poor pass down, 
coordination, and communication; non-use or lack of publications. policies. and procedures; and fatiguc compmsc most 
latent Maintainer Conditions. Finally, most Maintainer Errors reflect a lack of training. experience, and skill, whereas 
Maintainer Violations consist of routine non-compliance with standard procedures and practices. and infractions, 
bending the rules in order to mect mission requirements and the flight schedule. 

Conclusions 

HFACS-ME was effective in capturing the nature of and relationships among latent conditions and 
active failures present in Class A MRMs. The insights gained provide a solid perspective for the 
development of potential intervention strategies. The major mishaps analyzed were primarily FMs, meaning 
that many imposed in-flight Maintenance Conditions on aircrew. During FYs 90-97 there were almost 500 
MRMs in Naval Aviation, many of which were of lesser seventy and were either Flight Related or Aircraft 
Ground Mishaps. Such mishaps involve pnmarily ground and ramp activities and can lead directly toa 
mishap or injury. Consequently, the present profile and observed relationships only hold for the mishaps 
considered and cannot be generalized to all MRMs. Further, it can be contended interventions developed for 
major mishaps that primarily involving maintenance activities such as engine repair are likely not appropnate 
for ones of lesser severity that involve other activities such as loading ordnance or towing aircraft. 
Presently, an in-depth analysis of all MRMs is underway and it is planned to contrast major vs. minor 
MRMs, MRMs occurring during maintenance, stores, and ramp activities, and MRMs for specific 
communities. 
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HFACS-ME Summary 


>» HFACS-ME is effective in classifying 
aviation maintenance causal factors 
















> HFACS-ME enables organizations to 
develop successful accident intervention 
strategies 









> HFACS-ME should be applied to both 
major and minor accidents to fully analyze 
maintenance errors 





Presentation Summary: 
The HFACS-ME model, as you have just seen, has demonstrated effectiveness. 
With HFACS-ME, a safety organization can: 
- effectively classify (code) maintenance related accident causal factors 
- develop specific intervention strategies to mitigate causal factors 
NOTE: Because most major mishaps occur during flight operations, it 1s essential to also evaluate the 


“minor” mishaps and incidents that occur on the ramp and in the hangar. Such mishaps involve activities 
that can lead directly to damage to the aircraft or injury to the maintainer. 
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Now that we have some understanding of the HFACS-ME, lets take a look a the 
prototype database tool which uses that taxonomy. It is called MEIMS, 
Maintenance Error Information Management System. 


This is an Access 2000 based prototype, developed at NPS, the latest revision just 
recently completed. 
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The Main Menu 
Notice the five button functionality, plus exit. Placing your pointer over any 
of the menu selections will give you a brief description of that selection. 





The query menu. 
The user may select one or many (using the Multiple Criteria button) choices 
from the left and get mishap information regarding that particular selection(s). 
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Mishap Number [2 Class ofMichap = [E 

Date ofMishan [1723719 Branch ofSevice [USN 

ércratt Type rig Szciaft Category [ACSIA 

Mishap Type [Fa Location of Mishap [Ashore ae 
Brief Description 










Ack on FCF had beft eng of hot ight dunng chm 


A etter ay 





Contributing Factors . Level] = Level? = Level 
Work ch supy. anc COI faded to provide adequate tech cata and wrecedures due to complacer SU 


< Back 


This is the TACAIR selection. Notice that the selected criteria is in blue. There is a 
brief description, contributing factors and the codes for Levels 1, 2 & 3. Definitions 
for those codes can be found by clicking on the “Define HFACS Codes” button. 
Also, at the bottom, the number of mishaps meeting that criteria. 
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I> 





Select Multiple Criteria Query 


| Aircraft Model 





Click the box IV Aircraft Type Helo e 
next ta the . 
criterion that you [ Branch of Service £ 
wish to select ‘ 
Tl Location of Mishap 
Select the — eT 
desireditem | ™ Mishap Class” = 
from the drop T Mishap Type 
down box. 
Year of Mishap 


< Back Wiew Selecbon | 


This is the Multiple Criteria Selection screen. Again, the mishaps meeting the 
selection criteria are viewed... 
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...with the selection criteria in blue. Define HFACS Codes allows the user to view 
the codes for each level... 
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...traveling right to left in Levels 1, 2 & 3 
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And Selecting HFACS-ME Summary gets the user... 
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.. This presentation. Notice that the user can modify the choices directly from this 
screen (F-14/Class B selected). This summary page is good for pulling out data 


specific to levels 1-3 or within levels. 
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Returning to the Main Menu, we will now look at the Graphing Function 
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The Graphing Function allows the user to choose which criteria will be on which 
axis, and then... 
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Both the X and Y axis may be further defined to produce.... 
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...the selected graph, in this case, F-14 and F18 class A and B mishaps. 
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Selecting the REPORT MENU gets... 
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...a Choice of multiple reports 
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Al Maintenance Related Mishaps by Branch of 
Service (1989-19399) 


3s UnNiK 

Unsafe Supervisory Conditions (USC) 
: 33% 
100% 
100% 
O% 
O7re%xye, 
FSS 
z 6? % 
SIV 
OK, 
SOX, 
ov 


COOO-+--- 


# 


cj aC) = 


Unsafe Maintainer Conditions (Unic> 
4 33% 
100% 
OK 


GO% 


HOG O00 O0~ 
. 


Unsafe VYorking Conditions (uv ‘co 


o om 
6 O% 
So a 


Unsafe MaintainerActs (UMA) 


1 33% 
% 100% 


2 G7 VY 


Organizational 
tHiaztardous Unsafe Operation 
tnadequsate Documematon 
Inadequate Oesian 
inadenuate Processes 
inadequate Resources 
Sauadron 

inadequate Supervision 
inappropriate OCperatons 
Lincorrected Problem 
Supemisory Miaconduct. 


4 33% 


Medical 

Physical State 
Mental State 
Limitation 

Crew Coordination 
ASsGertveness 
Communication 

“da ptabihtiwF le xibilty 
Readiness 
Trainine/Preparaton 
CernficatonsOualmtication 
infringement 


oO O% 


Srv iron mental 
Eauip ment 
Velo rkspace 
4 333 4 


Error 
SatI VP IA IY 


...in this case, the Mishap Distribution by Service. 


109 


Slide 48 


| 33 Main Menu Be 





i 


bd ad 
— 
4a 
| mg 
q ae 4 
: - 
: ‘- 


‘ 
- wt 


: - 
eee ee ee ee ee Se ee on ‘ 


The Add New Data button produces... 
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A form on which to enter mishap data. In this case, it seems that some of Santa’s 
helpers may have run into some trouble. 


Again, on this form, HFACS Code Definitions are available. 


11] 


Slide 50 





The Investigation Tool is not yet fully functional, but will allow the user to “drill 
down”’ into the specifics an individual mishap to better ascertain learning point. 


Last, but not least, Exit. This button exits the MEIMS program and Microsoft 
Access 2000. 
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This concludes the Tutorial. You now have a better understanding of the individual 
error categories and their interrelationships in a mishap’s chain of events. 
Additionally, you have seen some of the functionality of the MEIMS tool. If there 
are no questions, please follow the instructor’s directions regarding the usability 
test. 


THIS PAGE INTENTIONALLY LEFT BLANK 


114 


APPENDIX C 


MAINTENANCE ERROR INFORMATION MANAGEMENT SYSTEM (MEIMS) 
EVALUATION 


Background. Thank you for participating in a usability study (evaluation) of a 
prototype tool for the Maintenance Error Information Management System (MEIMS). 
This tool was developed at NPS and has been modified based upon previous usability 
studies. This study is being conducted by LCDR Mike McCracken, USN as part of a 
thesis project for his Master of Science program in Information Technology 
Management. MEIMS was developed to address and identify patterns of human error in 
Naval Aviation maintenance-related aircraft mishaps. The Human Factors Analysis and 
Classification System Maintenance Extension (HFACS-ME) taxonomy is the foundation 
of MEIMS and is an effective method for classifying and analyzing the presence of 
human error in maintenance operations leading to major mishaps, accidents of lesser 
severity, incidents and maintenance related personal injury cases. A previous study 
indicated that prior to using MEIMS a tutorial, explaining the taxonomy and some 
features, would be beneficial. This study compares two groups of users, one which 
receives the tutorial and one that does not. Feedback from user groups will be used to 
further improve MEIMS and the tutorial. 


MEIMS captures maintenance error data, facilitates the identification of common 
maintenance errors and associated trends, and supports understanding of how to identify 
human errors in the future. The target audience for this information management system 
tool includes safety personnel (data entry & retrieval by unit safety officers, other safety 
& training personnel, maintenance officers, maintenance supervisors), mishap 
investigators-for data retrieval (Aircraft Mishap Board members, squadron safety 
officers), and analysts (from the Naval Safety Center, the command’s safety officer or 
one from its higher headquarters). This tool allows can directly lead to a decreased 
mishap rate and overall increased mission readiness due to the training and analysis 
opportunity it provides. 


Usability Study. You will be given a packet of instructions to guide you through 
MEIMS. You will be asked to make comments on the effectiveness and usability of the 
prototype system during your testing phase. Additionally, you will be asked to complete 
an “exit survey” after completion of your testing. Questions will include demographic 
information, objective questions about MEIMS usability, questions regarding the tutorial 
and its effectiveness (if applicable), and subjective questions and comments for areas not 
covered in the objective section. The study should take no more than 15-20 minutes. 


Completion of Study. Upon completion of your testing and survey you will be 
asked to return your packet of instructions to LCDR McCracken. Thank you again for 
your participation. 


Mike McCracken 


Lis 


Instructions for Maintenance Error Information Management System (MEIMS) 
Tool Evaluation 


Start-up 
1. Go to a computer lab with OFFICE 2000 loaded. Log on into the NPGS domain. 
Open Microsoft ACCESS 2000, then using the zip disk provided, open the file Hfac2000. 


Question 1: Did you have any problems accessing the program? Y/N (circle one) 
If so, please describe: 


Main Menu 
2. You will now have the Main Menu displayed with the world famous Supersonic 
Hornet photo in the background. 


3. Note the six categories next to the command buttons on the bottom right portion of the 
screen. The system has “focus” on “Query Menu”. Note the information on this button in 
the bottom left gray buffer above the Windows Start button. Place the mouse pointer 
over the Query Menu box (don’t click, if you do, select <Back> on subsequent page) and 
note information that appears in the Text Box (both of these sources of information will 
be available throughout MEIMS). 


4. Select <Tab> and view the same information for the remaining four command buttons 
(note, if you select <Exit> you will have to re-enter the system (see step 1 above). 


Question 2: Is the terminology clear enough to understand what each of the four 
command buttons does? If not, what could be changed to make it clearer? 


5. Select (click or tab to & enter) <Query Menu> 


Query Menu 

6. Note there are two sections on the Query Menu. The left half of the screen has seven 
categories to help you define how you would like to view the mishap data. The right half 
of the screen has four command buttons. 


7. Select <Aircraft Model> 
8. Another form appears: “Query by Aircraft Type”. Select your type aircraft, then 
select <View Selection>. “Summary of Mishap” Form appears. Note, your aircraft 


selection has a blue background. Review the “Brief Description” of the mishap and the 
“Contributing Factors.” View 
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Question 3: What aircraft did you select? 
How many separate mishaps of that aircraft type are in the database? 


View one of the mishaps. 
What are the level 3 codes & what do they mean? 


How did you find that info? 

When you are through viewing the data, select <Close Form> 

Select another aircraft type, and view some of the mishaps (optional). 
When complete select <Back> on Query by Aircraft Model Form 


9. Select another category (your option) & view the data. 


Question 4: Which (if any) of the seven categories do you find useful? 


Which (if any) of the seven categories do you not find useful? 


10. Select <Multiple Criteria>. Create your own query using two or more Criteria. 


Question 5: What Criteria did you choose? 


How many mishaps for your selection were in the database? 


Did you find this function useful? Why or why not? 


11. Return to Query Menu. Select <HFACS-ME Summary>. 
Question 6: How many total mishaps are in the database? 
How many mishaps have a level one category of Worker Conditions? 


How many mishaps have a level two category of Medical? 


lay 


Further define the system by your aircraft model (or select another type). 
Question 7: What aircraft did you select? 
How many separate mishaps of that type aircraft are in the database? 


Conduct further queries as desired. When complete, return to Query Menu & return to 
Main Menu. 


11. Select <Report Menu>. Review the six command buttons and their functions. 
12. Select <Mishap Distribution - Location>. Review the report data.Find your type 
aircraft (or review another) in the report. <Close> the report & return to the Report 
Menu. 


Question 8: How many Detached mishaps are in the database? 


13. Select <All Mishaps-Chronological Listing>. Review data. <Close> the report when 
complete & return to Report Menu. Return to Main Menu. 


14. Select <Expert Graph Menu>. Follow directions. Create one graph with aircraft 
model (yours and 1 or 2 others) on the X-Axis and HEFACS-ME Level One (all four 
codes) on the Y-Axis. 


Question 9: What aircraft did you select? 


Did you notice a difference in the level one codes between the aircraft (if so, what)? 


Return to <Expert Graph Menu>. Try more graphs as desired. When complete, return to 
Main Menu. 


15. Select <Add New Data>. Enter two mishaps to the database: 


Question 10: What are the Mishap Numbers for the data you are entering? 


Check to see if your entries were added to the database by Looking at the end of the 
Chronological Listing on the Report Menu (look for your Mishap Numbers). 


Question 1]: Did you see your data in the Chronological Listing? 
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16. Return to Main Menu & Select Investigation tool. This selection is not yet fully 
functional. It will be used to “drill down” a specific mishap or types of mishap to 
determine if like factors occur. It will provide enhanced analysis and training benefit. 
You may choose multiple criteria, just as in query, to limit narrow your focus (optional). 


17. Return to Main Menu & Exit the Program. 


18. Please fill out the Exit Survey Questionnaire. 
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APPENDIX D 


MAINTENANCE ERROR INFORMATION MANAGEMENT SYSTEM (MEIMS) 
EXIT SURVEY 


User’s Impression of the Maintenance Error Information Management System (MEIMS) 
Prototype Tool 


Purpose: This survey evaluates a user’s overall satisfaction of the Maintenance Error 
Information Management System (MEIMS) prototype tool. It consists of four parts. 


Part I: Demographic Information. Part I provides the user’s aviation 
background, computer experience, and availability of software and hardware systems 
used in the Navy and Marine Corps. 


Part II: User Satisfaction with the MEIMS Prototype Tool. Part II deals directly 
with user feedback as they use the prototype tool. 


Part III: User Overall Satisfaction with the MEIMS Prototype Tool. Part II 
allows users to give general feedback about the prototype tool. 


Part IV: (if applicable) User Satisfaction with the Tutorial. Part IV allows user 
to provide feedback about the Tutorial 


Part I. Demographic Information 
Follow the instructions after each numbered question or statement. 


1. I am attached to a command that primarily performs maintenance (military and/or 
civilian) at the: 
(Select one from the list and check the box) 


Squadron Level 

Intermediate Level (AIMD) 

Depot Level (NADEP) 

Command does not perform aircraft maintenance 
Other (describe if other) 


Oococo oOo 


2. How long have you been using a computer? 
(Select one from the list and check the box) 


Less than one month 

One month to less than one year 
One year to less than two years 
Two years or more 


OO Oe 


Vet 


3. What software do you normally use? 
(Check all boxes that apply) 


Q Microsoft Office (Word, PowerPoint, Excel, Access) 
What version? 
(Check all boxes that apply) 
ay 8) 
Qg 2000 
Q not sure of version 
Q other (describe if other) 


QO Lotus Smart Suite (Word Pro, Lotus 123...) 
What version? 
(Check all boxes that apply) 
oO 97 
QO 9.5 
Q not sure of version 
Q other (describe if other) 


Q Corel Word Perfect Office (Word Perfect, Quattro Pro...) 
What version? 
(Check all boxes that apply) 


Corel Office 7 

2000 

not sure of version 
other (describe if other) 


BD DFE b 


Q Other (describe if other) 


4. What software application categories are you familiar with? 
(Check all boxes that apply) 


Word Processing (MS Word, Word Perfect, Word Pro...) 
Spreadsheet (Excel, Lotus 123, Quattro Pro...) 
Presentations (PowerPoint, Harvard Graphics...) 
Graphic Software (Corel Draw, Adobe Photoshop...) 
E-Mail (Outlook, Eudora, AOL...) 

Database (Access, DBase...) 


DO 8 8 fa 
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5. What computer operating systems do you use? 
(Check all boxes that apply) 


Windows (3.1, 95, 98, 2000) 
Windows NT 

Macintosh 

UNIX 

Linux 

Other (describe if other) 


Ooodoca oO 


Part II. User Satisfaction with the Four Sections of the MEIMS Prototype Tool 
Select the category that best matches your impression of each of the below categories 
(and check the box). 


Strongly Agree Neutral Disagree Strongly 
Agree Disagree 


I feel the information 
on the MEIMS tool was 
in a logical form 


I found the MEIMS 
tool easy to navigate 


My tour of the MEIMS 
tool was very interesting 


The information presented on 
the MEIMS tool is relevant to 
maintenance operations 


The concept of the MEIMS 
tool is a good one. 
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Part III. User Overall Satisfaction with the MEIMS Prototype Tool 
Please make any comments on the MEIMS Prototype Tool not reflected in your 
comments in sections | and 2. 


The most positive aspects of the MEIMS prototype tool were: 


The most negative aspects of the MEIMS prototype tool were: 


I would make these changes (if any) to the MEIMS prototype tool: 


If you received the Tutorial, please continue on to Part IV. If you did not receive 
the Tutorial, you are complete. Thank you! Your participation is greatly 
appreciated! 
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Part IV. User Satisfaction with the Tutorial 
Select the category that best matches your impression of each of the below categories 


(and check the box). 


Strongly Agree 
Agree 


I feel the information 
in the Tutorial was 
beneficial 


The Tutorial made 
the evaluation easier 
to complete 


The Tutorial was interesting 


The information presented in 
the Tutorial is relevant to 
the MEIMS tool 


The concept of a tutorial 
is a good one. 


Please make any additional comments on the Tutorial. 


The most positive aspects of the Tutorial were: 
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Neutral Disagree Strongly 
Disagree 


The most negative aspects of the Tutorial were: 


I would make these changes (if any) to the Tutorial. 


Thank you! Your participation is greatly appreciated! 


126 


LIST OF REFERENCES 


Benner, L. (1975). Accident Investigations: Multilinear Events Sequencing 
Methods. Journal of Safety Research, 7 (2), 67-73. 


Benner, L. (1982). Accident Perceptions: Their Implications for Accident 
Investigators. Professional Safety, 11 (2), 2-27. 


Bird, F.E. (1980). Management Guide to Loss Control. Loganville, GA: Institute 
Press. 

Carroll, J.M., Mack, R.1., & Kellog W.A. (1988). Interface Metaphors and the 
UserInterface Design from Handbook of Human-Computer Interaction, (Helander M., 
Ed). 67-85. Amsterdam: Elsevier. 


Ciavarelli, A. & Figlock, R. (1997). Organizational Factors in Aviation 
Accidents: Command Safety Assessment. http://vislab- 
www.nps.navy.mil/~avsafety/research/orgsum.html] 


Department of the Army (DOA) (2000). Risk Management Information System. 
Fort Rucker, AL: United States Army Safety Center. 


Department of the Navy (DON) (1991). Naval Aviation Safety Program, 
OPNAVINST 3750.6Q. Washington, DC: Office of the Chief of Naval Operations. 


Department of the Navy (DON) (1997). OPNAV Instruction 3500.39/Marine 
Corps Order 3500.27: Operational Risk Management. Washington, DC: Office of the 
Chief of Naval Operations/N511. 


Department of the Navy (DON) (2000). Draft copy of Naval Aviation Safety 
Program, OPNAVINST 3750.6R, Appendix O. Washington, DC: Office of the Chief of 
Naval Operations. 


Diehl, A.E. (1989). Human Performance Aspects of Aircraft Accidents. In 
Jensen, R.S. Aviation Psychology. Brookfield, VT: Gower Technical Books. 


Diehl A.E. (1991). Human Performance and Systems Safety Considerations in 
Aviation Mishaps. The International Journal of Aviation Psychology, I (2), 97-106. 


Dumas, J. & Redish, J. (1994). A Practical Guide to Usability Testing. Norwood, 
NJ: Ablex Publishing. 


Gaylin, K. (1986). How are Windowns Used? Some Notes on Creating an 
Emperically-Based Windowing Benchmark Task. Proceedings of the ACM CHI’86 
Conference (Boston, MA, April 13-17). 96-100. New York: Association for Computing 
Machinery. 


L227 


Goetsch, D. (1996). Occupational Safety and Health in the Age of High 
Technology, 2"° ed. Englewood Cliffs, NJ: Prentice Hall, Inc. 


Hale, A. & Glendon, A. (1987). Individual Behaviour in the Control of Danger. 
Amsterdam, Netherlands: Elsevier. 


Hale, A. & Hale, M. (1972). A Review of the Industrial Accident Literature. 
London, UK: Her Majesty’s Stationary Office. 


Harle, P. (1994). Investigation of Human Factors: The Link to Accident 
Prevention from Aviation Psychology in Practice, (Johnston, N., McDonald, N. & 
Fuller, R., Eds.). 127-148. Brookfield, VT: Ashgate. 


Hawkins, F. (1993). Human Factors in Flight. Brookfield, VT: Ashgate. 





Heinrich, H. (1941). Industrial Accident Prevention, 2nd ed. New York, NY: 
McGraw-Hill Book Company, Inc. 


Heinrich, H. (1959). Industrial Accident Prevention, 4th ed. New York, NY: 
McGraw-Hill. 


Heinrich, H., Petersen, D. & Roos, N. (1980). Industrial Accident Prevention. 
New York, NY: McGraw-Hill Book Co. 


HFQMB (1997). Human Factors Quality Management Board Charter. 
hitp://www.safetycenter.navy.mil. 


Hibit & Marx (1994). Reducing Human Error in Aircraft Maintenance 
Operations with the Maintenance Error Decision Aid. Proceedings of the Human Factors 


and Ergonomics Society 38th Annual Meeting. Nashville, TN, 111-114. 


Leplat, J. (1989). Error Analysis, Instrument, and Object of Task Analysis. 
Ergonomics, 32 (7), 813-822. 


Liu Y. (1997) Software-User Interface Design from Handbook of Human Factors 
and Ergonomics, (Gavriel S., Ed). 1689-1724. Boston, MA: Wiley-Interscience. 


Manuele, F. (1981). Accident Investigation and Analysis: An Evaluative Review. 
Professional Safety, 12 (3), 53-57. 


National Safety News.(1975). Work Accidents: Records and Analysis. 105 (2), 
160-162. 


Naval Safety Center (1999). Aviation Safety Data. 
http://www.safetycenter.navy.mil/aviation/Aviation Data/Aviation Data.html 


128 


Nielsen, J. (1997). Usability Testing from Handbook of Human Factors and 
Ergonomics, (Salvendy, G., Ed.) 1543-1568. Boston, MA: Wiley-Interscience. 


Nielsen, J. & Mack, R. (Editors) (1994). Usability Inspection Methods. New 
York, NY: John Wiley & Sons. 


Norman D. (1983). Some Observations on Mental Modes from Mental Modes. 
(Gentner D. & Stevens A., Eds.). Hillsdale, NJ: Erlbaum. 


Nutwell & Sherman (1997). Safety: Changing the Way We Operate. Naval 
Aviation News. March-April, 79 (3), 12-15. Washington, DC: Naval Air Systems 
Command. 


Petersen, D. (1978). Techniques of Safety Management. New York: NY: 
Kingsport Press. 
Reason, J. (1990). Human Error. Cambridge, UK: Cambridge Press. 


Reason, J. (1997). Managing the Risks of Organizational Accidents. Brookfield, 
VT: Ashgate. 


Schmidt, J. (1987). Human Error Analysis. Unpublished Manuscript. 


Schmidt, J. (1996). Human Error in Naval Aviation Maintenance. Presentation 
at the Joint Service Safety Chiefs Aviation Conference, Fort Rucker, AL. 


Schmidt, J. (1998). Human Factors Accident Classification System Analysis of 
Selected National Transportation Safety Board Maintenance Related Mishaps, Chapter 
8. Unpublished Manuscript. 


Schmidt, J., Figlock, R., & Teeters, C. (1999). Human Factors Analysis of Naval 
Transport Aircraft Maintenance and Flight Line Related Incidents (1999-01-298 1), 
Society of Automotive Engineers, Inc. 


Schmidt, J. & Lawson, D. (2000). Aviation Maintenance Human Factors 
Accident Analysis. PowerPoint presentation. Adapted from Reason’s Swiss Cheese 
Model. Monterey, CA: School of Aviation Safety. 


Schmidt, J., Schmorrow, D., & Figlock, R. (2000). Forecasting Return on 
Investment for Naval Aviation Maintenance Safety Initiatives. SAE AEMR Conference 
Proceedings, Long Beach, CA. 


Schmidt, J., Schmorrow, D., & Hardee, M. (1997). A Preliminary Human Factors 
Analysis of Naval Aviation Maintenance Related Mishaps (983111), Society of 
Automotive Engineers, Inc. 


129 


School of Aviation Safety (1998). From Domino Theory presentation. Monterey, 
CA: Naval Postgraduate School. 


School of Aviation Safety (1999). From Aviation Safety History presentation. 
Monterey, CA: Naval Postgraduate School. 


School of Aviation Safety (2000). Aviation Safety Programs: Course Outline. 
Monterey, CA: Naval Postgraduate School. 


Shappell, S. & Wiegmann, D. (1997). A Human Factors Analysis of Post- 
Accident Data: Applying Theoretical Taxonomies of Human Error and A Human Error 
Approach to Accident Investigation: The Taxonomy of Unsafe Operations. The 


International Journal of Aviation Psychology, 7, (4), 67-81 & 269-291. 


Sheridan T. (1984). Supervisory Control of Remote Manipulations, Vehicles and 
Dynamic Processes from Advances in Man-Machine Systems Research, Vol. 1, (Rouse, 
W., Ed.). 49-137. New York: JAI Press. 


Schneiderman, B. (1983). Direct Manipulation: A Step Beyond Programming 
Languages. JEEE Computer, 16(8), 57-69. 


Shneiderman, B. (1992). Designing the User Interface: Strategies for Effective 
Human-Computer Interaction. Reading, MA: Addison-Wesley. 


Vincente, K. & Rassmussen, J. (1992). Ecological Interface Design: Theoretical 
Foundations. JEEE Transactions on Systems, Man, and Cybernetics. SMC-22(4), 589- 
606. 


Wickens, C. (1992). Engineering Psychology and Human Performance. (2"™ Ed). 
New York: Harper Collins. 


Williges, R., Williges, B. & Elkerton, J. (1987). Software Interface Design from 
Handbook of Human Factors, (Salvendy, G., Ed.). 1416-1449. New York: John Wiley. 


Wood, Brian P. (2000). Information Management System Development for the 
Characterization and Analysis of Human Error in Naval Aviation Maintenance related 
Mishaps. Master’s Thesis, Information Systems Academic Group,, Naval Postgraduate 
School, Monterey, CA. 

www.faa.gov (2000). Federal Aviation Administration website. 


www.hfskyway.gov (1999). Federa] Administration website for Maintenance 
Related Mishap training. 


130 


INITIAL DISTRIBUTION LIST 


Defense Technical Information Center... .......ccccccccccccccccccccececececcecececeecececece a 
8725 John J. Kingman Rd., STE 0944 
Fort Belvoir, VA 22060-6218 


PE AHR ENT AD. a52hce eeteceossccoavetdcanteashistsaccaiaiscoavesnsscoies-cscaaRyeaone’ 2 
Naval Postgraduate School, Code 52 

411 Dyer Rd. 

Monterey, CA 93943-5101 


Ma MIAO CIN Arse asics ac oswuauarvesscdscNeme <cssecs Meee ucawasics cet Pema Sensis eeeesenceueucs eemeeeeere ] 
Board Member 

_ National Transportation Safety Board 

490 L’Enfant Plaza East, S.W. 

Washington, DC 20594-2000 


MN AUN FAG Ole oo tsu sap -4 lead sz aaansecadessetetecstoaicsdscsds caer teases cease ee ] 
FAA-Office of Aviation Medicine 

Medical Specialties Division 

AAM-240 

800 Independence Avenue, SW 

Washington, DC 20591 


SIONS Ate) sto) Soe ot ob 676 1 eR, SY) eee ee ntests eee irre er eee te uN RR S/O 2 
School of Aviation Safety 

Naval Postgraduate School, Code 10 

1588 Cunningham Rd, Room 303 

Monterey , CA 93943-5202 


fero@lessOr MOUC aS PIMC y\.ocssc5ccscr,cecesssenenscecoteonss teeter cee oen tec cee eae ] 
Naval Postgraduate School 
Monterey , CA 93943 


Naval Safety Center (Code 10) 
375 A Street 
Norfolk, Virginia 23511-4399 


EOD Re Ke IVICC PACK CM eae. 505 coco oheesek wc eve eee ene oe esc ee se 4 


1189 Birdneck Lake Drive 
Virginia Beach, VA 23451 


bot 








ae th TD 


6/02 29657-390 



































































































































e . 
- - « 5 : "se s eo = -« . ° . 7s &% . eae ‘o. - ° ae moe “ee -« ° “<0 2 22 -« =- oy o- = . - =a - 
‘aaa pe acid a ee a ee RSIS te a aw Re rete eene © = S e «© » | = A a aes = = °e . se ° ° ese . = . . . iol con o eo e - eo a «8 ee - =" « - a ee oo awe er - -2er 
- are a een Saimie Oe Gel ao pene Cccre peen = Low s2 - e . Sid - ° : 4 aoe @a > ee of ow? ° a a rd - em = - ee er eee - -- 
La ome o am pede, | = e« a eo 3en er = a hs . Ps ee = ~ wee - . aoe . ® oe ee ” = « - es 8 == « r 2 seca - «” oem “26 - «= 
- Shaws eece con a Cee. 8 ee alee aes aat . ee E S a ee ae . . . cS e eo - % et eo fa e “ee o «8 * e @e o - «© we - en Se we « wae co - = wen 
correc. Ute meee « 7 bea! “ aie ia! A oe ell a foam aa ‘oan a La - - F a e = - . w= © © = -« 7 tee e o ° Pe © of @ * e . «wee is 
] ee One 55: ig te ae *- «= © « ee cls O . e - « = f ata 0 - - . e . e wise we os . - els a Se) aelel mee we S eae mee ° - -* oe - =< @ . - . 
- ghar = oe teh 2 Bes - ° ae ee feet o- e- . - trae eee ee CS . o = . ~ omy 2 pe a ueen a = a aore . “7 ee * o one . Caoe@e e e o 7 - e 2 aee 
et J i. ee = fan, eee Ladi Fm wigiusens - - - =e ‘ee ; paar bd G - . . . oo - . e e ee .e . - - ee . - . tena e - o ma % © —-a2s 2 eee © - 7 2 « -- i iad 
eo oe ee eo eee - mr tei =a se ae gay bist 210) ay ie eoees A . ene - - .. « = te aT em ane & = ° « -_ =e ® * ac Sen ase en ~ oe ° - Omen @- « ° 
- om fe “- S reel ae ee - Ow wis feet s ts =- a bim0 Ce" hae ate A ‘ed - - se ee Bs e e e coe = e matareres a « 7 e= os =) ~ ee « ee ee ee . oo - os an - @ @--eem -¢ - 
- . oi o= = « >» vis «= ea a8 «6 pee dad aaa Ot sa a | a 3 ee ba Yo = o. 5 ang = - .e0e arate asseg @ e © © w . op "m= Some o- -«- -* © a - © se « 
oe . = mins meat lowe ele a - ow e ate aa Meats * . - = .° « -o% . ee oe oe . . Ce = = “ a e e qae er ee - - «2 ee oe ee ey = mn ae ° esc = e @-¢6 
iy, oye as ao "seme ‘ae ‘e ele ve sd a aie ees bel o~- co en, te aus be . oo . ? ° e e ss s* . . wie wre eee . . i at ae ° a a) sweae « - ee ee) ae = e - - one e = oe 
mena: = Gh oO aie hee = bf . S e . . ene =n . P= o ee . es. Ct eet fore = ee. e 7 = = en e a@ e - oe ° eo #8 wae ee ee td os eer ete a ome - - o> wr 
=es = a fee ee are Seen we © @= asae Ld - ~« = r wae) s . =. ®©en en. a ae me cone es - < a Si 6 He = oe aokioan . e 5 aac ee ee een . oe - - o- se ono @ ad Ce ee ae et wae 8 ee « 
“+e Deer ~ aig a = 2 we = Secs f - sy 6 Pa ape ‘i : id o. eer 8 maw hehe e . ss au oer et vansers ae awe = e on ah eae eer 20 oye cee 8 we oe . - oe - Pr ° = - 
= i. is - ene “"ert ea oe - ° ery a = . - eye ieee tan metas ais id A . . Si vels . - ane e the . a = Fe * e = -* ° e=8®e pe soe - wewaemee ae a 26 ee ee ee os - oe 
_ eh A otee =e wis mle? sate sleteco eee ie See teh oF Pee ene . . = e . . = . P : Re my - < a aie ef oe - . enue = - es tw == © cao © . er ae ee ee eee ee eet oe 
“am se we = a6 miata ola atn me cee ame Pielke ae o- Par ay gine eiwae 2 ar ee ee | ~~ ¢ ra ~ eo en6 = . he Sha sie ee a ier oe Pa ~=m e =e 23 = we tes -= - | a ee ee ee ee ° cere s. & 
—e epee : eeene oe ae e vo wie ° Fe i o> me - & aan eegcaencoete mae eS eee - - . - a e es a = = = 6c4 se me) on * o 2 -* -=« ze. e ee ee eee eo 8 « - Pa eee er #f - «= a: 
- e oe eevee o “2 ‘a e- - « =e cm 96 O08 © ma Sys gr ae Hee --e - = jeer yg en ~ ° - < =e oe aoa ‘etimes euu = == = oe e= s se ee en ewn © ems e - mea 0 = -we = = s err © = =" 
o« . “Ss e@teaeewey, = elaine - -—2° » = can P aon <"o8 e Me selec ore we - = . . = e eo aie 'e ry ry an ad =e = ace se es ° eeecen - e=..* ser es e - oe +7 - 20 © 28 Ce ee ee ee e . 
-. =e «© ~<-e - me e - ss ¥ - oN OP OMe nea a 7 rane # hd bd - - - . i mea’ = * e ee * 2 mle wie @ een ey e . - ee ee ee ae Y «fern , ? a a ee) ee ee - ar o 28.08 
one ake a wit orate eae an. aeeeetm a = : oh oe em Gg ay ate ie s -~ = eo. ° © ace ~ Ss 5 . os = ee . =e « - 2, « . re es @ taser awe near -« - «8 = - @e 22 o - @«—e «#2 
oa ef = Cores oe wes get a on mee ne a Re = e - we = ae Per A ee s . ° e ~¢e ° a s e em «- ° = ee = Gre @ ° re «28 a eae == «6 o w2e*e@ - - ee - owe - i i ee Se id en2e a 
= oes PP ot, Sow we wate ee Sw imi oe ei, ae - “=. "aero . =} Z = . e ° . ° anime an oe a ‘a aietie Pe mcrae fe atiee ects egtse es ott Oe OB le a ee) e- « es - @ - « - ¢ oe emo on eons 
ee aa om. eS eames a ec ps : - +s wre e sien e e ‘3 ~— of ee © «© agp = = ey A my x - a om ri a 5 aie cea inn ae ale e 5 on © an -= . ee 2 & « Yt aon) sae owe om = * efe8 -s @ eo? “eo ~ 
Ramtec wee ~ eae = Oa 6s as a beatae eed SME gers ae Vigan ee meee -- «© nie an ae Ly - ed apa we e == me Piao . e ’ ane = ms wae et te os =n 0 se 8 oy ee a ° Set Bete qe ew td ele se emoee “ee ee oe 
@weenwccaceec. = ~~ m «oe = Lilia a) ia Sas ace ey a “= «@ o-. eo ee * “= = « -— « . = m - . nie es Pa aie mts meme . is ° aoe nonwes o e -_— * ae ee! o7Re + -@ "- eo @s - = 2 ee @e +f -@ oe 0422 
As teen cc. 8 es *. = et puae "e+ -. Ot ew ok see so e oo me e or “ - on e e. Py ores RS ey ai —_ We aaieca, ai a wiatel ashe meres e ee C8 ete- oe e oe i ee ee ° cec sn - “eon - erc- emer 2 ee He om: 
a) Stvier oee) ae eaeteeee =e ¢'e'S) 8:08 = eer aes se ene Sim) wow, to eyatarene rae Care, - =. « =o 8 = -* e © se *%& e em * e a = @ 2 = = aw eeoe ster p eases © = + £ w@@ = * “se dt A ee ee ee ee 
ee or = S) ig) aes aie i "sc @ e.- eexvane = OS @ ae © Cane == aid Chere ew as hed © - - . ee . . - eo «ne waa eg e - ~@s ow e ° - . © e ean . - =e ee ee ee ee ee) nae ee ee) ae ee sd 
= at ven Ve SSeS), wie ele cin Ter ee eee a Ge: al ace Sree - =. mites mor . = 4 . © #6. « es . wt one aie rae 5 ee ie _ : = cates em we ce os @weanrswausece &€ F @ Bee a= ee ae ae ee wee -« 
=m, eee, Meme P.O mieiele. sla oe Se wma ee hoe = @s ee ste0 = oe oe =@-em- bod -'s -2 es ete # - , on vee . ee ee any ue Py FA am Py i ane re na onan 4m nae aw eee @ C=. = es 6S « ees © @ wee a er er ed co @ ee ee ee) 
e ay ra Oe ae: ae ase - ean | ee i a i rae es fw ee = aotei ae eel . > - "5 Pe ee a AP . 5 a e wataten a “os Ah aap Soe ee ws2ee we. * “ ee 452 or 8 ee ee ee eee - -. == ee - 
era Smee co vee wee aie Pr > s« aw pao Pests “eae @ . = « vase Suan ere eee . ak » oe = . e- 5 ee = e a e Seren Pinta es - 8 eo - -« cece eee a e ee Wh er = °F e@@ e-~ a © =2 ee co 2 ° . ee ee ee 
end ee ee ome wT etm eee Fae eS mage = Ya) te) ee . Pease et) ad Cc See tr oeen anaes we x a ° refer er . ' 4 e n= «ee ° Se eer seo @ oes 08 weal inne atec "vs o's 2) ‘ele @iea. is 4 en ee ee ee ee ee | - em <= «oa - 
"POP ee mace ce eto 8 om. = » pel aed I elses . < pA Darl > a ine mie Kaew e are hee . = 6 a= on ‘ ary ° Prac - e e ey ae. oth . - 8 -a ee | oy er oe Fe ee ee Ce ee = e- eur 8 = eowila mee eme e208 a@aertce > 
bedhead (ie es eee el wei am. ere Oe ae ae ge oe me e eye le us 7a Giwsiei te phage) a0? o = avs = 5 ee = - - fF > © emae =a @ « netet Senta) (ese wone-e = Bh wg BHOe - 8 206 Ce eo ee =a "@Peme er ®e * ee Se © oe e208 - © - = 
«- oe ef eH mn ee. ace pa = eel el al ie Slesetas te Te eee A ™ 2 ee “6 =« Ge tae te 8 eh e - . we en 5 ame « as e@ eens =? fe e a we um 88 oe ie e ~- #2020 € @ saeee =e -8 ®e7+2 08 = ww ee ee 
= 58) (eis tes ce FY Buraes - ps ‘ pig ne = 6 oe ate arte ae a 7 age . —= =e =F a mee 5 wa aan oe te 8 ee a iaance! lew etd tent ine a meee win) a) are os eco «mm see © wetter, © - « @“a@e0 t= mrt oe =282+ @.4968 @F «< 2 #2 gee ae 
“*-20e eae Nes Sie = ome wipe ofa Sey Sie “O Wiele «ae 2 is (untae a ba SRR SL; O- of om ee of fe = . ems e Ce ‘ in a“ oo = serge ge cone ess age . ee oe ecee “es ° ~s - ea eee er 7e@®e e2%e@= Gar: 
=e. pe Orie Here “- ee a i boda Per . a a ne ) me Sree heis od e . . — =on are = «ee ee ee @eee = e af ee oe ve e a eet a o m_e we ee eee an 7 248 @ ce @ee eee a ee oe ee e . << oo eaacee td 
Meenas fae eee eni eae ee . aie So 8) SS == me vegies, \earae Sexe e, a ° e &e = © = en oe = ee cee ee ae eur iS = EE Sede ae on*—e oe “=n s@ © 4° aoe we gt at 2 ems = @ree - «+380 eae e ot oer =e -- 
ele! celence c2F\ 6 wee + ee See ee age © ee re ete = pao ae ee = ome os ne e = . te Fee - ne Si) igi lieereiace: cai, -0 =n iy oe - oe wees Cte ene teat OS - a ee ee ee eee) eco eS oe 
eo es wo wy ee eee @e Onigine fee Ree =e ° - me ee ee a Pres ore ies we e 2 on i war geaeie en ae ea aaa a wa'eem «= - 7. - e@ eee Se =< ct ef eae oe s=me-+ = = @ere ee al - 
ee ee Lend (awn e eet Same 6. a. “- 2 @== ese ween of O Se Mae 8g ahetel gee eae x ee » one ° see a e . ee e=> rs a ee. ° - - ae, . ete om 0 ae en ae ee ee ee ee oe ee ee ee ee, ~~ e 
POMC m, cw e ie oom Sle actete ae ome =f e@ Fe ten © . ee ee ee © a. © pe: aan es Sea! M “se = 6 om « « =e . TWA auee " s a = oo aes ¢ ee ae en ete - = ee oom 2 te OOH 70 © 2 me eo sem ores ears = tere em = 
eo un he Nbend ot I eR J) i eova aa en latea ar al Agta =e eS Oe es 7 ee - =6 ~~ a Ce Se we 5 Ppatd 1) += bet} es @e - = . ome se. = a% e ie eee - hee ~* eos F wane - = on. - eof wane ot o a ee ee ee) =e a ae.  ) 
em own wa. See) e aa ee diate re, eee nie eee ewan Le bz) SENSO pe! oe =mere : s -% ee ee .¢ ec oe e * ome Mel Lewes altielie! 6. ip e wee oe & ©,8 = -*a a2 2 "#28 « on aoc eo 5 8 wee 8 Oe ae eee = © © © eoowe : o 
Ce ee coer em eos fet Eis oe © - a Piha egies CS a ay Gieveriaarer oe a eo =m f= ro ae es * iets - a A Cee « e mi) fate oe MEAN “cain a eeee - we s ~ = eo. =m ae . ee fT 2 ee sereces re ee ee ee | 
t= Seeicct Te, se mae we wee St perp eite aha Se _ omer " meee ace Xf on sme eee 4 ome eu CRS 5 a . . er oie eae ne et oe Oe ee 5 ee as “Oct Pee = @2eon oe emnes CH Seeet= C8 82, Few 882 gm wSeay yes aoe gee e 
Mat egn tw oF ee we ww we ee PN ee Frey enact esaytake s ea tf 8 mee 6 th Ome SOS an fe Se evel Bowie) vie 26) ce = me ot on ee . “=. e = -? -. * a eo. wr200 8 29 ies 8) (eee ous = 2 “eo we * O33 aes Oe ee eee ae ee OE Oe ee ae a 
lied bel ee "swf meme aw ee ee » Ue - ee a ac -“e mae o@ oat eee © om ware ee dead ee @ en@ @s ee = Ce ee ee ee . aoe . ones es = « - ss oo na een =e « eo. een ag ae = a este 2 = ere ee =eeoe %, -2n-¢ on =e ree SS ya" Gases 2a = ee 
weet Shimane a ee me am) NaC eeerial gees ee tee Olney ier neat’ eleanor Cee tata fet ae yvarccon ee e Fert ec 2 io oe = eo ea a a career reer a rife eh wigeau! meee) as eem ¢ ij aeP © Gun & Owe © © Cae - ~e%et «8 -s 2 ene merom ec%® fe SWE sete omen a One -e% ef & “os 
a ee Me ee we Ge Baca eM se) Stn ee) etme meee ie 6 one oe ewe ee Ro ras taser eoeeaie ea ve == = ot oe ee i) = e ’ ety are =a te a esgee s ase ene « " ‘ ° Le oe © =a ote. w twee we = ose ¢ OS Owes ee ee 22 - @ 82.5 «© O68 SECS Fa = - = 
oe ¢“ne es "Pe. mean ates ve - 2? aw . om —a¢ ea pil ays eee enn . - 8 “e+e 4 ; —:-y" eo s@e to" * mw eee , - so mh e- . "=68 rele ane © = ePuste . me ese e e - ers . ee eae = *§ oe oe oe ~eere es - e-e a me -ee 4 oe we. om 02 Me oe eas@® @e@aewseenvats “—@ =. ee@e ae en 
ae ts eee mee ete tem , we, ne akcriMeina ide ee OW eet tk ein) ee eee oe "A. "8 tm Fp Cee ee gt pm =a a ane Secheteeci re = er a er a ee ee ee «« #e ee ee er ee ret = 828 See te eee se ee Mee & oe 9 BOR “= ae ~ ow se" 70a8 
oe ¢ és ». <8 Stee ee me ge tee = ele, 36 e Pe #7 Oe One se af. wee @ Oe aL vee . 2 87 8 wee ot, 6 er 6 gee 8 @=508 Py én ar) ean. 7 aes vow cam wee Mae eA eat . tet OD -@ 5 epee en & e =. > Oe ee ee ee ee, ee ee) = @« me J fuenm Smee e = se Liha) a 
°e se Oo ean © Hr ot Orn we a / Oe fsa Tepes eee as 8 oe Se eee oo &. se te - 5 z LL ete aL) “a Fase, = ease oy «7 ® « es = «© pe ye oe eer ee oar < u~gne Dart eae ei tans . ’ e =e ce Mt ue ote lw, Ot we ecm + =p = 88 ee ee «2? =m=t@ee oe e228 ®on “se "geet of te 2 o P#S = oc: 
ees . cone Se aie ocw " * nme & om . 2 8 eer & @ oe a © =.tere oe oe Wee Wists < % . wa9) ee. - 3 o- or mera awe i6 - ae miele a a. omne of ewte cme fee om ese =e@er - sete em «¢ @ . se « es e ‘2 ©« pe seme ee 4 ea es «2 "28 ee + on Pe oe ee oe) 
wee mn ccm Foe MT Pe taee wc. ec @e 2*.0 Pipa ha 9 = a ere She le ale as evan! Udit, oS es edaneg: * e« . Hae etee eee O ome ar eg POR. C = ‘at Vel wiaaelwe co wilasae = erm eos o=gs =~ ee © core "e-em @ Om Peer a ee eee en ee ee eee eee ° 
been - @en a» papins ~~ Se So we Copa p Gieore Se sae aa dae Ba west @e.n .2 See ee os © = . . 2 © o_o -> wer = 2¢ ° se - = eon * eo #2555 .* we 7 thee tn @ < & © 8 'ae 9s ad ee be ie aN i Ng al do oe 9 nese = =) 9 2 2a 
Ree heey. "© Gem tewe = "oy - "@ «© o _~ e. ie ee mei aiven) ee B20 tame one ew te © ane = Pe Sr -o a> ome a oo een om eee er a a ie a = aea® ee er ee en ee ee ee ee see en gs @et 80 0888% © *8 eo °2e Cause 8 eevee oo # 
se e8@ tela = ee Pena AES * ence mewn oe a cen, ° "otwee Gane 8 -#- = ‘ea= he pe a e.° - &e = 6 atese Co Pe ee a¢ oe @¢ as . mp we 6 ea eee, ee ee - “n* s'®sS=sze@eee ete ~*@e.ne ees eewn Some een *2e¢ @ oa 
eee 8 me tee Mies ligt oY ot a Pie Pais one =™ SG «so oe cent aed fer we = 8 SN Ripples were = be . sem eo . see. gente e =arere aa smy ° fe ae er ee ee nen oot et OF Pere OBO Om OF FS gt 8H ete wee 8 ae.ee CCH © OHHH -Or ow ~& - « ou Cero «as 
° Bia clone Gyre ore Memamnans nay oe © & twee = #@ see 6. © = agree rae - us "eca , © ote Pi paee == ante 0” os e. - <0) ea aufaisivetearaia Sina #: scene whet Selnlce = = ee “oe ef eomavet wee tts ate ee 2e- ee ee ee ee ee emacs 
SNiaiwacs © aemns ae ree fl es oo ees owen ‘ aoeoen oe @ea 0 . wetwlener _™ - "9 © te, Oe oo = et wanes ve i = ae care - oa an oe os wa ee “ ere wt eee wees se -~- Se -c® tog etavon = @ oe ee ee) oe ee) a ee ee eee) 
ote ceioieter ts = 8 (9 Mae laos Mam eg eu Pe e8 Ome eo D4 =e « » Mak mae ee Aa EoD =e = one ere « . = ants So re oe a 8s as . earns aeeias walt asiacstiicnte caee e-em ew @ Ae vette = ee ee ee ° oe *e@e,¢ ee 2 ee ee » ee =a"-a@ « 
6 8 he eae eee 7 9M 8+ te we os pea e Au esunmsne seen «a. GC me aeae eg Te eaten ee pee. cm mee? tag OTe oe * tte « @ » fe * upam wee ew & O8n ws « & ens == «@ 8 sore of =e -— A= eet tree gt CP Owe eR Om oO oe Be twee Stee 68 8 Oe SS eeee to bso com et a yee 
sen s os me Se) 1GY Fee eslel nao ae Gen ae eS enliia=: Wadea! sae a Sat BMY mee 0s ep belo gnc hoor eter eee ene ve hs “te entee os in . enue eae St sacaiad (on el deel See) ap) moe, es 7 eon -= =§ @€aeme en ee er ee i eee a@2.ee 
ed Ce “0 = = pee =e ciatavwere eae = a Re gees éecn A on @€28 6s «© mente) ke) (weseye mMatese = meen es e © © eees.8 e ee . mares akaseuta bee ore ae “5 ae Vemehagialal Oar xae CoreecroFomaacer ey Pe ete a ee ee ee ee eee ee eT eee ee eee ee ee 
We wes <5 ae a iuarstc = ©.6 ane, wa, - elmer lala omen ee © ae wore ow, o Paneer hel me MP AYO!) @iag ™ Pe tee ae eo Savin, Smmieease . is > te « ae ee eS ee eme “ast e¢@ ete ue) ana e =e a Pe er te « emt te 8, te ee ee PPD oweee SF 8 +e coMm@iseeeo® oa 
Oe MeO ain gigs = OOca Rw Te lwiie: elm win © NTS Cems © wag eg PORE DP oe te ode “2? >» . = uioce wite ta fe ee Roe eee ee ees) Cee == mn ete ae . Om aan ee 8, i o@ a0 oe « om se = 88m 8e coetenms ce mee me wi ae BP Oee ete wow eee PTZ See S228 we a FB fo - & @ 
ee., Slay) is pee PLS OLS epg) ee plait a a ee ena Bie Gwe ene Wee ee Len Si Stifesters " - oe rd o e280 * ‘ PS eee lg ea nade oO @ = 0 © myhun a ee ee ee ee eo ee Pe e = fa © 9 © om ee eee ee ee ee ee en © e228 a@e % oS. ot @ 
we me Be we Wisc ne of os re CNS 8ne Wem ene ax pes Faeoe =¢ a3 tities! ae ae Aes . . . Pleat] ee Pa a ee ee | n 5 ees emus wash ca’ @ e0 (Geese i ost eR tes one Pe ey ee oe ey eo 
PO! Bo ehenies nt e- 2° ¢e ere tain eraee Aue fF Rs wee one mises kcelsus ae ™-e e.=% Bet gy oe ers nase © Weert. ae "em eon ge wa nee) semeSrm © me i Me eotee aXe gel eae alaee es des ereaus ‘a eee ee a) - = - « e «© oe sae @ ewxrteeemen Carus Tee ee Pee en eee ee 
CS O05 a wae a a elsie caer . Se ae aoe a ee Se oer. = eee a it gel Dah LLL g | ree en © ane Sat) Tiesia; e900) @), 0 e016) ciel tees Ne Hl gical maar “epte ee 0e om atm kt eg we a ee ee ee em e@rneemence Oe oS oe MS 8 eR OO 8S Pete NONE Oe OR Na OF Fo yn Oe ¢ 
“1 > anew) ao, eee” 8: AMEN ws cic ae ae cee eee et BL Malone! gel ag gine) 1 ce tay, = stem «= Cae ee aes ss 8S ne 6 av es a 8 ° te . ee =e % 8 = = ye 8 +. 6 -- 0) we Seow em oe en er) * ee ee ee ee ee ae 7m © Mf OMe eee 2 Ce ater tt 20°76 
we = swe peee a S eo = a © Ope ge, As act ae ved = = @ ue we <tetes si Pees asses ge J te@an = ao bed 'e heed - e = = «et one res oe . ~ - * «08 eat - e fe uous e c ee ° See ote a am 4849 “ot SP a terse Ce eee ee er ee Se 2 o ors 2emte ee . 2228 § 2.222468 oF F2 @ ee aserea~ 4 
~ <'R=.2 a Oe Pie er: “\Selnlaiiersign nS Saphire nad "eae & ove Ps Bef oe =e wee ae. aes Tete rao zi heard =~ = ¢@ os & 8 ra 8 @ wees 5s men. te etn orf as = @ se sete a eee 4, e058 ee ee, eo ee | eteaeam «@ an ee a er ee eee . 7” @t22e « OO ed ee 
os 2 ae Cy OCP a teed oe “Penasco PA perf ee an om des Oe mee = ene sme ne Os Sas'@)6u) ce ldgit ngs hare Se Owe = tee) eum 7 - es = 6 cies mu ot 2 apeg Pe 0 8 wart wf enece ne gtjenesBade 8 2 8,8 tte eee © Oh OS Ome eee eget oe emacs wnt eG oc TPERBe ote ow OT re sf Oe 6 6e we Sea ee oe 
CCP eae Me we ck eh ee 8 oe Be!) ae OO a ae ce oar © (88 PF fem 6 wey = Ome, Cee Gwe g are: core Sou tes eS bie Jel Ne a one tem 8 7~ “ = . = eet - “ge ec sem os © 0,0 @nt a0 Og, de ss «fe OF 8 oe «ene eee © @ ot oe So © eR Myf: 8 Oe rE wets ten OS ee Ont we wet «fe @er oc ag8 © oa we 
oO 2 8S Ss 0 wn hos@ecn ttre! ot PF 8 Perce as ce © ome ne gies == 0 © Gee ns en fm @upemre 8 wrweee - aus (te a se ace -” Fue Rte a Obras © se &  semreg 0 Ot © em tore gerne 0.086 8 0 Ow OHS BO ere = Bee ete Beem e sel ee cess soe / = OHM SSO MDa get 2a 2 Fe -~ SOMen = 
OO Rye Oa go ae mo ane Veewre = “ tem ew may a F © Aw oeEe of gum & oe 6 80 age tse 26 Seem 8 ogee o8e was me a8 - a 2 - © we Cra tseme ee oe = MO wre ewmec sarees “SeMOORDZases%e @ - 2m 
eo tem wm nim aac, ca e a0. © ae oe 6*@ 2.6 S08 ‘ wwe s oe oe e a +. rs et = at oa a ona e ym ens wt,o8 eee 0% eRe 0 © tp BE ees OF OSe ease | me Pe ene Ow © Oe tse om S A wm ow One OPER BF NO 8 we OO HEE POSSE Be oO OM Te aD eae 
“a we mee OES es © Oe b or8- One aye adit las io) Cr area = 3 geue . ° . ° p20 er et ee eee ee eT) ~ qaqnee 6. eeimence a oe VF see omson s £ Pfam wo © wot emorcne ‘Seve © Bet tee Oem ope 228s toe eae © CSB Oe Seehe nee 
OM 8 we & eR trees oo oe RS BO Mae @ OS.c.eies ene\n @ sees . efectle: Messiaen e gf was 7 oe . sh Pastas A re oe 5 ane 5 meee Miah ckalgiea Beene eevet C= pt OCR ees HS 6 ww ew eHow Oe Gree ee CMa wet soem at eee 8 « | F wHe SP ecehO ee! wut -« eae es ot 
kA u ae NSS le oa 07 » prme.e a) gh ouwge! Somcaereeiecmun ast coins z ome © OF « be oe 8 one e see Se Migs Pepe ys Pas tg ow < Sib ae) ia, eheeeD ieee ry) ee ee ee on om e i ee ee | eee aw Cee Oe Pesemeetseorcate S882 emnee 
Steee coe ee ee Bp iugete, =om® Siqews a5 98ee  chwciee, Seca s <0 a= nee ey KG) we te Nene a Bere eawerane Siecle ' ‘ Pies ecaml ees ae Salen ae en ot ee en ee ee eee es Me owe 6 ON 8 ee BRC eFes sa «= ee ee 
"lene aa od a = Beene eee OTe ae. Ciwka ye) we ce < Le Deer ae a"=" 66 2 3) 4° bt Aub Latch dalla > =e ‘= a = aie ala = e = £m a tw ee obom on 64 On a Oe ee Somme oe FE we 68° Pome wt 222 fe O- 
| nO wee Stace . Cini aan geo alte ree a wuts @es eure g C' ee oes ve =° wea te ee. a2) 06s Cac em cats ri aw 2 « oe @ opts cet oc sal alel ee oe es 8 - i fod - ae oe =e BS 
te 8 on Oy ox % om AT - e e378 a Aire ee srcs a eots cwere ew oe gw . ee = cc <6 anawie z aa ai welcal apras eo wao sea aen! ete te Sanee gues Oe ee ee ee ee eo ee ee ey e woe © me 6 ee ee ee | ee =e 4-02 -2e-a4: 
ra bs ap adaa) _O O78 Cate ames8 0 memes emi e nee wee ps e - par 4*tme tne 66 a s each)» Sa ao Soe ie ee ia abel Sosia 6a etek ew es 8 8 Be meee et wow es FB FP ew mew whe Oe 0 o omy o Oem, © 8 Pec we ot SE2.29H 08S e2--%8 
Mee On" Orem se, es bad ba oe a er Sa ake shel ates aes peli , eon oe ost * lek se aa Pr ery a cone sls ols gal ® 6 Sica wee) 0 6 an! om see ° oo ee ee) o-tre 8 82 we oS C8 SB Os Oe eer =e = Yee et rates = od Bite oti Pd * cant 
= ie o amt 92Fe pie = tte poe &ea¥ s 7% wf . we . =e @etee = 2 @o 88 eos Be © evece Oo ee | B25 te es eo «am © < . » — = mee 2 2.0 Oe ee oe -2 #2 ®*e ws a> > ore ~~ = 
Mees wee o.8 me eeene OP 8 Pent © © 8 ase stor sae “oe om ges ane e sex ’ " a e faa a.Pteterte ec 0 2 wnt ene = "eo ae Se wee ® ven 0 og OR f pS8 tenet Osc "SPC Fe SPO era Eeeel MMoy 12 Se CHFPBCetES == = «2e2 e020 
2h © ame Wipe ee eascerene Gee eae — %, ee ee eee AeA o. = Precis Oy . wee 5 5 nelsG ute Frise ones fous Sep eticpl ir - z oo. Page ce ee ee we he tO °, eo, 8 tate mre come et ee-e Sere Cree eens © SS O.2Oe 0O SO OF 2 Gm © OSS 9T8 ORO ONS OHO OA OTE 
Bt iabdmaatt Mierpes © a wat 6 :. baghdad Taman Qs ¢ On 66 a "6 5 ee008 eacheorere ° Sister ee a ¢ Cm oot = ° “s) te ne wo = © OnweOto = &*, - cers eee oe - 2@ % o uctee ee et a we © os eter % ee a ale tg ed in ie ace ag yey piety rome ee Pals at be Saati ae eel Sere in oe - Tate See 
\tpecs act = . roe = e@.t eye Orie = © Cen @. « : = =e ne 3 gah eae © eas = Seva . neg meet <@ =ncere gee © ed MH Pr en ae ee ee ie =: = ee e are ee « ce - = 
ween 208, S.m78 ae degen tee es «3 an as Pac taece ' *e@ we ° 0 he - e. 8 . ee = @= pPhecee 4 mee een asta tes we 8 we ia . t-e rr} e . hs - aoa ge A ere bs ra) gear emeen Scar . Ze eneeert - oS O19s@ OoFrt OF @ Ogre eS * Eten erwe* © oe 8.5 62 e y= © =A awe e pg elle ». 
age ed ah at aaa at % aT 8 we 8 =e ==0ats egeecs <¢ Ore. 2" Coe ie se ee oom @ e “= «8 eo + wt = £8 © & w aseraetee t+ a re oe ee « 2A m 2 O Bue re owen e TBH Fa 8 Opt © ust 8 wet Peer Fe Ct aeew na 8 OM are Oe igtp Se cael ae - Po 
Wn sss eeu © yoa ome . ac Teens PaeR ETS gt eo te Gate = tem @ 88 are e = 6e 8 8h 8 wp 8 8 ww pee 8 Oe Ota nny O18 out 88 P8 08he Fans §& CAP o Oghth wt oe © ater Smet C8 0s Cees a o8 ef 098 (= HOR wee (26 = e.e= emer. = ea ean eo 
5 bopper cel . chad Bap 8 ow igmet eu! at a Tale me ne em sa “a sabe © = 8 oe Ven 8 aw. .~ . s ae! we =e - 07 20 0e @ wmtme ee Owe wee oo fwnes — OUP cre OM 18 em FR et OP OS OD tow trun + 8 oS HO EG a eo OD Se ee mes or © ee *- ad om ee 
> ad bate delat « waeeg wae Gh dict roaa OX e Fat fn = © seme setae © .) 8) veer "een ome 8m 8 RB Heee © + ap as Seenmm Gntetss © ge Cem 8 gad “8 7S we aye © 8 Oe BU grt © Cm Am eM ee OO 8 Fo Oe ee fo Og TEE Sey DAM OW evere = teowe 
oc hal Wiha Cama laa tae a" Satgign 8. caucus @ ckagave pees - © "66 «8 econ © © ee ome wee p= & ete em ome ohn ete to See CR 8 aee Otten MeO ZO, OG OP Cet TZFaThe (WSR 0 82S 8s oo Ove e oan ee OO eae SrA ieee - 
wnt 6 8 ere, fo OF Crarass © oyu creo ons on - e®-, © e = @ Ce ? ae re or er er eer ey ee? ie ee * oe ments mee 8 - e ens 0 Cn Oe Po re Se ee a ed ee en en en EE GET e,0 Bema O° rea a lc streaming ra? 
<< ataronios Pu = SO en D Bleep © we 8 Carty toeee cowmwepe 18 Rois eam ee e ‘a= eau eas » ¢? et «.° Rakaun (6 och a eiieLe . ww tes alneleuniale.ece) o.6 on te a eer ee eee ee eee ee ee ee =e ee ee Pe eee oe aT 
“ eee ete ne Manet ee oe agi pars stints SSeS sa! in om we 8 8 © Beene ome 6 #8 988 Oe ce we wtp e® 8% % at pe MBps OG On + weg Ce Be Fee 88s © Rt ceren we FH at 0 Re One wtye O12 Oa om SS ota e-aw * gee oR 0 Pee ee Oe Se Oe Foe = ae o=e © « Se © Oe « Hae © oe oe 
a haa cita ae CR es bee Cevatve 6 Seuoveem ateare ease = oe F © sete ome meee 6 @e o ee Otece avon mf nig o Ch easneREnte ae &e ye. emet tae 0 C8 temas © SF Peau we wee 2 + tet we fF Om Cease oes Fe ee ee ee eae ° ey are 
Sa COlre erMeGer, © me Osean esi» “atewas = m bd Se OP ws oe oe e = . = ot es et bd casei reg seine 4s ete CO tee MP8 he wee e ene inte © OD O88 = wo mere n-c.8 Oe SEO © OWS Or? OFT TIS 8 en OTA COs C2 EB SRE Sf 
jm ~ @h, aon © wens eiee tec elicssasaran te fae « u staves 4 os 6 © te oF ergeee « oc Me + wee Sicetaurs sa.nte ae eaeesa se Or ee aa | - 02 ee = a FO e  nrt oe 6 HOR Es Oe PP Ow hoe AHEM TTT 8 CEM ee &48 09 60 e Ge Teme SSE SD © OF Oe 
“e we om mh, Eh aie aael | Tieeredetne ee eae el kN Vital ener Spuert # © e o- . e 7 -« B <éleinia wt ESO ut Cg aatenie orar tyr ee © ew Ge BS Peempe Ae Cio! 58-2 ene LS) (RATE et Niet ea ere re en re re eee oe era 
= e Fo © ems 5 ene Orem *% 8 Be alas abs a Or ge “"e6¢ ° © @ aseepran a ~ =e ot gts es eo ~eaet a ee Ba «@ & t=0%-e ete Qosp~ee oe a ee ed eam. Pam GF ernp O18; OOo © OF One HOM So COS Samer eOwWTs emo ea @e@e@eaneeee+ Feeee Faery 
« e- pee fie - a ee ent a6 “== ©€@ © « - & *e= oge a name a we ee a % ee eo peace eax til aa Bag Datos 8 oe — . *8e 2 SD Oe8 ©0 oP M2 a *) 6,8 me @a,e of OOD om * © ee ee seagate =1 
4 Se tn ey * wees elie ° s . "es aoe e - tn & «neeso@ee on Suge "8 ae . FS 3 notes etace = > aes we weeeo® e aaa woe = 2 emo ~~ 0.2 Ce eae CON foe, ¢ Le te MS ow © oe A= ssw cawe -*e re Cd 
2 nt we om? noe a bebial 7 Rte C2999, Meee e see = © 6, oe 2 = @ Se Pe oe a ==) a mw | mst og 9 oe peay soli, at « Pew * whee 2 0 Oper Oo we Owe 00 Swille SE1t6e OR OA) SFE THE ene BL agee © CaN OHO. He Cw nce eS 8OMe m= = Om SEE ORF | 
© ae = ot SoMa em sD in ese et we anaes eae OrP ee eeeeuey tm e ee % swot? ag ee == 0fe + ey ee © cent? a * gemquee. crowgre ws "se 22% = «908 Sia) isGre (oe.( m= Sethe CR 8 28 OT Sew a a « (4 SERS 00 6 6 O88 Se eT ST Oe ee peti deaats ms 
9 rages 0 Gactora Woe mn ge fetes Gary aNese gn cm eon ict ®, a oberoe ae tor ee gate SM Cae ee ee eeere © Slane ater 8) fyi w © 8 OFe,9r OBS oe 7. e860 BOO 010 OS Tew wo othe cee 0-008 OtF o™ BOSH OF DOME OSES 548 6 PDO eee * States 
=< > += ame Oo Coe i ey Phe Wd agee ee ° mene op east) @1ee A © wonae aa onm taal oon 2 enee rege mw at e. - e m= 6 sect * e re i iS eo: pr ture ce tecor BOe * = Ree My | 60 OEM ot coms Feo saeZo 0 ORD © Ro FYt FERS 3 0 & OS 6 02S F= = 
° a ee eve Lt ad oe Gey ceOlne Se oe re 4 te t= ~m a ce :, eae ae o ma e rap ae age % 00 gh wt erodes @ On8e = © eentage +60 Or. Omg eh we em Oe = A8O gy we 0-8. CMR © OF wrth - MHORQZaM SO 4.2 DEO? 6 EE. See On oe eee 
~ i tntaeeehoe Le I el ah « . 6 ‘mene greue 0 af ’ Saracen tier einie Gage) cungemcetay So colt ol. plan papel eneynrs(® «ae fc ee 8 Ones | ett an th Oem.8 810 Oi we = oP SOR OOD 0 ODES Teme C18 Se 0-0-419-S S-Bs a es eT 
: A SOee Paws neeOnne oO © + Ges we wo ae cee Paes al ple aeailao tsetse) nie re Sice wl lSebagh oe Pore wr at ee ee ee 1o 88 COTO 6 tm OO OOOO gS OTM alee MTS OP OO Sy = © ase Oe Os Oo Bum ae Sane ae ane et 
> ad tJ = © @ =08 see.e wee oy @ 6 =e Yo eee aye at err gy varias te ate Cee Ve -e een oo, en ‘eet Ce ee ee Cee eer’ Tir ee ee) ee eee ee ee i tee Fee Feo we Sey 
WJ eg “erte." > © Bo puBraes ° 260 coe @ © aie Ve lceuets acre = e6ine a 7 at me oe A Sas 00"4 eeen Ane pecoamerareah Den P-SnOtts B, Ome teme Sw POD FL epene oe OO ~ SE wot eeet o 92 Cees es * 6h teens 0 Oe OF “aedee ahogiees ow. « ow Sy 
4 ara On 8 F = Dg ByA-Beretem, = a es ; elem? Be gue Cuter ‘oe “en ” o wre © fF" *6 5.08 ane ete 3 om Bete ate ee 7a Be a eee eS e — Oe aac Om Moy tee x et Fw OE ORME * C@ we OPE 2.) TOW ETe e+. =F Ora ~* aS ee -@Wten 8 @& ee 
=d Oe Oe mee y Bere ce meee, o Bars 6 we bl tect eh il Nal at 8 o Wr tee 8 wee weret o2 OF 8 ole e seme alieden om os "0s cee - = tebee oS Wie FUE nw ce My 888 oe? OG 8 8gnd Ae we Ca weemtee 8 CFR & OM SEO 00 CMe FO TET] Kota De eee ee ee eee Shee o eo 
5 e. TESS WD Peey ane Dp oa i ° * se de Fo tearnce e e : rf het oe Lee dhe = oli i slant L! 3 i ° te Opn ee a COR Dale OD A 8 © Fe Poe OSL e oS COO ES ~ SR OD we OBO = ee 2eseee 
QO Pe oe ae omen, oe en ors -°2 @ &, era iacaie A . ae . ee) ee re Pe ry o wet ee '0 ee ee a et et et coome 7a so ost s ° mele ty acre Te eae Ge Oc ealtRT cone O=amme re Use ata maar a 1161s eRe a’ wen[ OR © th eeenee SEER eaeesiaca ER ONEA 
-"aPee . < ve a) = = a .. een e . ee - sO9e. ‘ aad wdptae oft ous - alee = cS 
= Phe OEP. O06 OP nm, Oper @ a: aa 7 rer) a Batniseetarers ® = O90 NEP toe ar ow} et ae ae at mi be eewe? gee ° on eo = aiseals ba oe e . aon, @ s ee omens peep es owe eebiart ae # sailieue oan BP net. pa ee ae OOM © 2 Ores BMS Oo Be o-ny) OS © SE 9 Y,8,OMBSS & a. = &enom 
= C9OPN Tee 8 OF OM FED oh aye een Petes hey Os commsatgent ee 58 a elite Soul erate oe ose Bi sian aleusi=seagce) (ote out sine Drie! arte g a Sean sue coakpert ie smlas © Ma oem aie oO Ae SAMA DS -SHODE FEED OOO © OTS He ane SOD 
OQ NaF © a2 soneeetnt « oMiasunvy Ouoes © qr. codam ce tse go '8ns wf tee Mwenwemse@ onto rs ip hod tt ali aa a ce Pee Poe yeaa ry SF ecsgrree €,0 eB tee 82 Ont S18 Oe Ce ews mae SOs © FE 8s are 7 WO Oe OSS = - TON ORs - we ore 
Salve f° tees we P otte ee ev Bn Fv OF ty Or 0099 un @ ercede cre my tse sega welts @seene Pay a tee cal atic oo Leaded z pot = ee @ eoecere “wu tyeee etoger ee OS Het He Owe TAM 8 Mee © 8 Tes ROms oew aoe Ces ea see ere seme Oar oe eee 
-% @ ae OO8 0.8 Ae ew, Ce ay mateo set "4 2 mys Ges Galen ee ae eee eal ee td = ett 2 wee Bonn pe oF 648 dee Bee Ge 288 “Um “P1RRcerwecgcorse = eo _ = awe ae FRAP os oped Weaee ees a Srsin te egal eeaiee on agen Seer Oe oe om OGume owt e OO 
ee foe, art oe ©, a C i , Seated =m 2 of Slevaceuameeieneee e dh e «a Cabs Pence AMmeAteeses Fem, POET S-On o & iwokmettrd, OO 6s 18 SoG HOn 81,8 Ort = Qereerm™ ee e-ow ” 


? 
beeppeahthehL AO or meee) CBA Sa Cee renee. OcOCRt Mee aRyEeS. Oe wre S98 AP Cormee OF ea Peete Brent © 08 © OO © PS © Oe O2 WUAas « 0 woth 





“SLA. * 420 foase 





ats om as "Beer gs ormte tre fh a 8 
































































ee - —_— 
ia. . i ere pase ‘ pare ere aeie' 2 a ‘fee ore ype o Pe gtte@ects ong 2 eee ae wee © Ao (MAR Ree 080 a-NMe CPO OLR MOS 6g «BF © 8 MEAT ODS @ oS oBHO8 2 SSS OT POLS HOGS - © OD NGOS 2 a8 
09) Diane ead i ee ee Sbislit allio vfe cece eapack (vase vO 6 PIG RaRtE ate Breese OS BOF © = Femta FEHR Oe ORAS See OG 8 OR ESS COS OR © 0 Bee OO © EN Oe Oe eae = 8 ene a 
te oR A the Bs pws etars ae ee @ 8 engete® olla Peet eee 2 ot n® Cm me @ ONT FererqQe 808 8 ODT GON HF S-BHE © o OOS 8.8 4m "SHR OMS SS STMO SAS ey 94. e5N's a ED OD eet 
ae asece ca ewenerae o ate se $0m oy esses gtep m aco x Saree Re ats) ian) heen & Bee ar Os Cem Ne Ce PERE Beets + © aed © p.21e, epUOTE wErnyeye-s ge © 0 Oe Rhone 1 2 e OP Care = 2 O80 Foe ser nes 
har Mle tahe Coe Serra eo eM wee ee fhe te (8 Oreneeans mints oe ee Bte Serrte eo of eRe ote. Ere ees = aeDEMy® A 8pt ate we SP wo Oe GO o'ah We Oo + © 2 © Oot et om Bo OE OU BOO ® arenes 
stand, eel? Pf refs = on @ . Bed ea® en er) o-V » werent ae C1Bee te a= « + an Cah Odie oye ose teePReregts @ 8 Hehe DS Bem SP O8 Fil Bp Dern ge OP BepeE ne SF OM & F-24868, 066.9 010 an -8 S SH 2.4-E ROO Oe one oe tommn Seee 
Patel ee stacey nists et en ot, A! esos * i efian tee ener oh Ge Piscacese "0 we teen ca@ 00-8 SO © URE RAD Bo OF weOF © On? © O HE = Oma ee, Few UE MraysTEtys ese <— GM ee HS EAE OMS OO POD BFE Bee | 
wales tilde a rts om Fy8 sm Eivod “ise ermsen 8s sqieen wy 4 bree © Le wnt we hat em ne tem CeOrte®. Hy ae Care RePane CORE 2M 2 OOD WSS, Oe mee Sone whee o2 Se @ Jae tN cael pg Ici Meare Md 
5 CBee Be semcane 6 SF oey my <0 tes ee en i vee . = wae e Rcerre nepe 200°? S.molFCl gO 0 geen eg! & we Ge OS Ole HOME Sw Oe Sm ERO Ce Se OS ORS © OU te H ° ps8 C4 Se on. 
ea a ta Setancan LS oe ee er | 9 Ge F228 g 40.98 © aow gad pre Ce he ese eta e-6¢ J . ©. e we “22 @20ee...0 © ee © @. e 

fer een 0% 9) s o- "em, 16 88) syneghe eo eee Bceg mecoe 20 ty Gh oatees ae Opt, 069 were 08 ene One Carew ae ah yA SO PR Pm a | ANS poet BOrm-6 Outre sO tee * “PAA "Stee © ws Oo oe ES <> OE  & 

oaee Pe 40 5 Bary ~~ 2 Se wees ~~ w= eqe we se we 8018s i agate ah © Se wmeeetsgeet® a terRtee® Baws © Cededs mod @ 0.20% Oe Dm m 6.9 82 9 an8 Oehy =" F SOO Peawee? = OD eam e Or Tae 6 C2 ED © m2 eo 
machtecesce e ese ave ow - —"pere.. 00% 20e @ 06e% =ne e o . ° reo4 Carame eS © ev ome Hie 23> = eee Oe Vehwn"he beet eo ere wyo® F atpaey ee Cehesonre gt SeVaebeeehe of sy MHNE-9 = Om ee ee een a cae ain aaa arom a ee ee en eas 

Be © QOsOghrwy a: of Pe ae ey a ®nee@ @ en, x mie is e-aj}e =a en © aS Sse SBum Boe Megegwemr sy tench tse wy © eke ye = FO Be SH aphes & Mgt VEE © ODA WES om fea\*h _ baeomed ae Se oe: 
<= come *- « © eke ah Sisteeprs ore.e¢ ccote be big Pug ee led achat etd < Seana 2 oa ape pre vet ieee codes . Mee ceamzes aor orm eee mo 424% San © _. OPES Be Oe. OP WOR PO OOm © © ODS LO CHL ADAO4 © COU SSE OO 6 TERA SOTA 
Oe — moon neler © O28 5 Pntere s BOER © 0 ele otf. my nee Siac aah : 2 = suserenee PRB rope Sraseses®, "eal 6 o SU ee oe eT 2 ee 09 © we bags ert on FOP ORO me Sot ae eed eon eee ee ee <aas 

P& 098-6 Pens ys ee enOete FOrtse ee © Shutare «= quensy ge 8 offen ws oO orate . eres! e ae hg ane ota Be cecsece «0 wt KE Gap MotuD O»Slager crag 4d am er ee GP DO ppm 0. 190, OREN. Agora ® © WA © S82 e OMEGA BUOP « 0-6U MS OISYSS OS HYSS SSL OS Sane = Oe 
RD Meaeeye C00 Hn wey - "®. CBR Pay H 2000 Bee i Net C= POE ee F0898.O e008 AE oe 2°O— gs mee Pit iy pce tind 4s Cals Somme veces Salt 95+ ae at toe i eneg@steagr shee ° ae one aseay weed ote Oy Ch GS © OMS SEP MONO SS mt Ayes FORBES (SOO OOF ROS MSF Vy C= = BRS Sm Os 
P ttre, © ace’. - me 0-2 cojed ss Ph OFC e 606 oC awle Sree OOuahow-s..07 = saa Nae Se age a eee Ses ace dial od AS yen os mae 6) ia 2 e 08 N28 Ged ORM BOYS EXE 6 BG HrOe os he” UP MMe 8 age? Ete OE NTE Bem O-MOs ~ FS2 4, MOTHS “oe © S40 6 = = >A 

PP © 00+ w Ctet me, & ' starers OOo CoH mre Fame SEP Me o=M OOF Rnrettwe rf ce bigs aS sa 3° as ike Sytem wey ee = ° ome = ponerse sei ~ pater se Fs a So cachneas arse) neg Mote wo 18 OPO Orgel he Wp COge tes _ OF © RMER © CMR © Mtr! © SS ORS © OES © Oey ight? MATE SF en ere =" 
Son 0 At Re, woke rere eco’, H coe tna: abaten Sea @ < o-8.N teak, Esegenshiletun pen (© bre iecetere te ee « x ieieus one =e iese Peg raentan eae be - ae seve 8 Serco SS cerca rita eerel ee awe fons 2018 0008-08 GS poet O RT w! inane & a ROE Ig aye Pe S 8S © oe & 2 = 6 Bewwt e 06h 06-2 22s © 
ne oa Pars 0 OD NF megs OF CheeOne = Hh ess 08 96 lie % O56 Gate a Renomees sf0% ays c =e ° ey Z aoe eee aaa ouye,S Siem sgePeh me *o @ 8 Peat eFet 20,0. - het & 0 Poy Do OOO are WO © OF KARE S 0-0 CMOS, SSE OE bd pads 8 Ne 
au foare O eewerra EMSs 6 Core 958 OG BE FuogGsde Wee BooGiguzats setme ge “> - peed oe* B POL epuoe?. Agee efeSoe gouge eee e coSse TD Om 228 ame Pree Rome Wt Fox oe SO By we,2 oe 6.8 @ =. nee ee fo GREER: OOF) 0 =D. ORT aye 622 Qe =e See S24 2 


hd dite del ot A aS et 


. 
~ Reecectet eta OF Med aRd trans © © © ES wh yh oO AM, 5-BeO2s PIE Bool RoHy POS —aGhS SON OP RgP- Cote Op? OUE0. PA Gym ipese? WE~USTO IS ae 0 me 0 me CAYO Ge ae On ae-a- ea: 





“tn 302 @ vogre Eirgece OR «nr*gecte — . te mgte Bn Be 4®@ 











“ete © Met. OeP: tre or mee? eat eerad M4 , . ee ab 8 Bytes cae se Oe aot ® HOt CoN Re o/Bwe © Olas Me Selic Sar yore estel oP .0 G oOo Che Oeam SyOSE O48 Saeh MSs om owe © 
Powered v4 ee ¢ @ eetee aos es aivis 6s med © 20 seemed aye a ee aera ee © OM Pangeor =oOn! Reeract PRU GR ee oe ng 6 OERPP LE Com Amy OO OHO O-Sie Cee wes PD ED UGS Sgt ST Melt aod = 8 M40 Og Gr OWS © °F ER? 20 Om 
ey © s@athbe® « CODES tee Mey 4 ce » GP Orere- Bad. CaP EaP) o) T08 (RSP GL" She (ghPRR ym Sansa secu eS Oe Eee Oe Oe ~—ab ae Oe Oye +e Oe Oe Be 


Beh pm 

¥ cass 5 OO tee mS YO + Clee ew + Oat we SE ].s 48 © 8 Sams S08 aes On® 2 OBB SF. 553 6 
te cer.» ee bepr aay bere yes : Rahs cetiee Seats ww yp fae pe waeem “yp “aptniae ye Te ee ee 
dar = ae O° 6 © Optra yes Co fat,e O5 Hoe wg Oe OOO ne 6 Oe wet © 8 ONE OE eS C0254 2 GOES + 6 8" or 
: Dwancaegsere FN Natale Ved me OmP-0 O96 0,0 S28 8 Sym ame ep me COP -MiO-2ENeme We 8 ee CFS Som FO A e- 2:98 -& CHEM SO, OR 9 SOS] ROE © 
bale, sexe eee 2 ome MTP FOOL ® ache a me Pe teREdse, Cytrlet Mee 20 FB @ bo %:O CDS BEA Wartro 6,6 S OueEL® \ ROE SO OG o-EOP ODM TONE OO Art © 





re ane Vout ae eves i GE Relea ete 

: ay 80.499 Stared 2 ' 

o BePeeg cpee-8F ese. Os use dat, boil bated 
< OF Sree Be evens etfeas@ae 3P, 9, 2 py etn te 


Pravtecm seme Pr00d" gee ety aap 





Or wo OUPnt Pam enes bes 0 

4 +#yrae e 
ese Fa PaCeere~ 0.05 Be PO BOe ory @ ae 
“© BM acB es B® cisen 



















































































































































. : *o ots Bone oy Se eete. 05,9 Ne ecm . he @aed g te-O-n- : 
Ame= det-s an a 2 ' ° “¥ ety® Gn eves ae 0 8 ee mie Or Cie EA PLe OA Ry Ola OM pre ORO Ole BOE Oak + Re CP gts GS Sy QTEENE OE SR. Ree & 
/ sa eae Sree ocean 8 ey at ae tas Seem avacerane a aeaece Dae Same 0, Estee mega GEMS! myN Oe SpR Emm Free PO Om Op aT TOSS! Ce eS O08 ONES oo PUT Ee a gS Oe Dee SOLED 
7 + 6.o ow sBace - se © %elOse ¢ O& ¥ crap Sete n fee | mete Ree® © oe oO rapegereered Pa Vey ee © © ante PO 8.21029 € ae + he Oam* DpH OO.@ 10° % Quam en? @ pO: §-2O- Er 2s Om. © Cems Ceo uO as mips aay anne 4 7 
otzetere eat oo 4 re ve tee Om Pee Racts=or y Per[Der wee Pla deetney wydprs Ader a, Shee a OF Ff 11,870. A7M toe One Mho-tneg to 0 © seRee a Ses ee eas <tee ee ene an ones Cie re oS ee eee ead 
WORSHHE=oe = 4 5 1 u <> s aeey ns = Ae _ e e:@e a a? Qe se Sr eae | eo She, 8 -~ me’ < Saree m™—* a Ce one om 
Ot eg onsee ar duced Bevan de odes poh eae core See So ges) iS et © mien se) 2 S0R8 Se, h Salsas sPeu oman Nel eeeseee ch 8es Sect Lore ctseNe wnt 206s 200K Mee Oce | begoneec cy StOcEsMse Hee GMyTE LMR whe eyone Cap one AD qa eeenae Co kage Suu O25 O08 © Ure tee y 
eau eee 9 Ba Sad oadet | MMe! © cents? =m 6 stace eeaes)| glinpueae ‘ eeette rasa ce ern ckeca esd conegetspusturs eles aesershe SOM Saslene O91 e=0 tol, AW OS 80 cs tCeerRse. OED m Oe 2 gene on OEP me 4 OO Ome? Oc wl Rabe tere NED a papa 
Red a acd Ta tal 2 ea P pus > OyPentne uns PA RE ee Ce eee Cor UNte ce ectbtp lla") | 4) mete tear a 808° Lum ArPo fan 'R 00k 86 Chm UNOLEA,S © Oa JU N.OHO DRED © OS tr OIE ERNE? B8, WIE SO Oe AROS © OAS a 
Peleg Pepe lata cad ds © eton "0 scene BOR ire ans aes Jae aed! zaiteeees nak Rau ancamnnecn Smears A © 8 8 nS Raye Ge Oe Se MOUED BFS Ca Ge I NOS®- VERO? O28"? Mowe wrt « ° (Stee ame waeeree' vo. = <a 
Beers OPaer@. beens Sratetcentare ll : 20 'p ia te Seu "eéencls Ps Sara Reece Past ac@el ne ccsgeubta0 atl | phlihstnec@eess! ona a tone Ro enen ty gh OOP Se WP ORA8 At Sa Ren 4s HOD FURIE OT OPE Hye 19 © HAS S WG SS. AQrTD Pegn | & © Syne es nga} Sage wae. 0 es ee ee 
Peer at Se Pe eee Okabe ht. a Overtemee etham ee og ate yee aaten Brrr 5 gts de pet =: SBeee Ady L:0sFiere-asee @ CAL MPCHNAS OR Ew Bhelp Renee Sraagas mee © Atom sm -rae me oe Les ME Nar ew Mew ewe Pane ‘ 
uae OOo om sR Le palGh Bev wy ope Oe8tOg aes or O0y © 2 te “wt 8 AB ctecet: ote 88s gt - baie yee ie te oe Be 8 8.0 Me gepre.t pce vy e Han? omyh tho ye OP FO GRR tos 0s OA 64 SNe ae ae $ epee eran 
phe watt UPS oO PHP eee) «60s 60 ae, MQree pe tcnaes Mosteoetecss nes + Sc eeeae zee aS eS Sota nae cele Sa) ave © Pe Beem -aeteAlG sl Me VO aMlOre SrtrO i038 OTD O° RON ome oes Pa PIE ayy om = mon me np enti 
« PSs esOaeg ‘ : : f : - of! ore sOuvasensac® 897" Cedreen: —-pwnrsfansce Seer oé : 2 G8 eee Ope BORt we SE! M a Tete. 018 nam whe es Mle. <P ae’ RPO we = = RE  aneploenn, edind leita Aaganl 20 OO 200 G2 LO aw 
noes pe mete 5, > vs us Baers et Sreeies erst nb vais Seaiereuee F059 DE Pr o0e-{8Mernste cuales By) MASLTe O.GoOETI MH HH LNED Syn ey nePage OEP ne na ag eae OBESE FO 2 SERN I Oe eee a caval Bae wee 
A o.a rane, 9% $e sree m abe 2 qhebaene ote co TG aM G08 95844 i ORE patron Ko ee MOE eee ee © 2 Fre A ERI & Stel S eee 210 Bors 2: ern tees avenue on: Ce re ona talie oC a aa 
man: adey ctwiysrpret? ae ‘ > 7. . » 7 A azoere eh nage eetowt® Sep De © © © 8:0 tee Weer & 6,8 °S See onus7e me 0) ogy? = 
Psa bos eer toa PL Pr STi ak eee rl ee ring oe emereeet) One SI niles toa Sere eR Ce Sy ge +00 | at ees as peared - aos pee es ee wae ees 
a Caben-peyn secere-gae dee.u Peas ere? . we te ; af @re aptewetgec®, See te, §° gtd DOs St FAN Ome? hee 1 Oe Dl . ° a bi = art atagte 
ofa ee seated eon sera li A! apy a LES raped fe 0 0 we tst2s Kin Oe} Weir Ose o parm r 08 Sutam? @aenee® onem pet Sarre lig Se apse eget 2 SOARS ED SO ST ate omen 
— 2 =pot Rise hm owe 3 o > = om rd Ant WHOL. wi gee CTO m8" @ Ohed Biase Lone! omen age 3 bes 
Suse Sr rian: ri a ausea"cpaee lease: a ae wre ry sevens Roenksae® % cay Pee nepaie eete RiP + eet CeO oO € We 9.0°8 OS Sante we ete pe SdFaw ou HOA tee-S2 ee ORLA S ChE 
. Popa trial E + Stee mon ‘s.o0 tedy ova. teoe CO ee sree tI me perase yrs aru enad e ap aveneae tr ieuieas rote nape Holmes page beep eat e here ag Seer. ae oe 
Oe gs 08 | ase PNL AMEDD v0 8s : € i t taal HS ssenee aasye ome 8g", wae face! ate tOeree- ¢" . a bare ac 5 aEaeee 
Sree a *ORgP wo otes Ney SEBO teeacowee ag ome sot *eRt rma 04% 00 Sr tiriere =H “1° ppeedeahed ALP te Sree ae, ese sane pray rts Se ee ee ot bgt ere orhre ameter ed yom Fa ars ory — ee yoo ee cape sver tot 
. ole! 2 7 : « 2 09: @2ge ene? 3 reps i ra ° * ¢@, Bg otee 2 Re | > y Epi rR > 
Meertse eapancaerpar trees. aeecriese: O-NasBrG.N c=Secncee oN Sobsan) mee epee PAW ca Red Dome Accs (ectgan'’ “Usece lay saul biG ne sacete Sotetuhe ee Vaath ere AAs A eile arog ae Orme ie, 0 oe cao. wale eee Re - em ueten tees Oe eA ean Sale arn aed 
S70 .OE epee Rosana, e inak een ete 8. Fee Fai Ss Rhy Shes apd 06 "Bop rtn te ce ne LETT © aft DetereP theca mdeunee” leo cc ee te ih gasetivem cee tw 8 wd atetrriaeei eve obs, Deb Shake © Ome oe bHh be GaP Ferd Orlr er Rabe oom! =B CipngnT Oweh Pa CPOQeLE BE O'S 6: taser % Rd a Gare S-eme =: dg pe aptmap ty eaten” Sg ee Sq ane %aaettenyoatrt 
Map etee cs fared os ae ae Tee este tate POS BaDe — oPQteR Cm on, aePom fide Be @ oe Pom, by gets Cure ePte—gem ass Notas on Fo@ acGe C900 “herek Oty U0 S000r prety ® AgtsO? ~~ LS? OrIRrOOM OD OLE Hep -0s tary SUH 2-8-2 © MOBS ONE GE NOM ES, © Ham OO Sa Sod! OANA a > Dene ane 
oF ota am .e/ 5. Sep agerer Bad o8 ee Leewmws, cee Creda © Se ePA clam aeRO mete HOE fg 8 NPED ORD cat tenes sen sengassaete Py en eo B00 ER p edd Omit ORS Ie Btw OG PO. 0 98 yee eeneth GE “ep om One rE S ong we tel po te 
PBs aed OPH. 0 © ob. a8he Corns® on Pe a8. Wir eena, 0 come Peecet=Gy mote os Bg che Beret, BIH ae tf Bo © Bles Tey 01 1Nine ws Fo name gto sete WPed %, Ory Behe RONDA SPs © BEM AG, C7 TMs @ ©-2- ORS O ET-EREE CGO O 8 © scmpmase / See amare 0 a0 aan 
ware: Foer pe geeehe Onan ivene st oF %eclh asenrgo ance gas My : NoGemde gO Bate enteenen, CMSs OP rum pbimicet® 1d Reve wate racrd a9? Pyreeae TORS ‘eampap BO°e Qenger Gere aces 0,6 © OK wya-® ove Sean one Genl= 0-0 aa Sree p>-s eS Ae 8 See ae 
> _ Sas Dadar. oe DU PPMYT Eépite. -F ect emed! fan 9 otkne som Cabal pee ea . Ten he 98 PE ages o= Peed © me ght Oh OS. Op eneee, Ap ectey ies Eqhen, Gusee F16-th4* OO. WSL BU mee Gberwe: oe 7 pais. © BEE Yoo e «& 
A RP otter terre © we e268 -Peggie Ate err eee.) . “pe Qo tstp ee 8 eth a C2.Pas 060 BOP tes . © Remezectot, ode My tEM Ocd, ear? agen omen wen te ee ee Randnare mone Oe OO 
> Ch ee La = = * VOmeDt oft, 5 "ae acem cet 0 oly . . Be os ah on 8 keoe® oe ABpod neamey b Qa oOO0 wd « ga GtRepad Hr Sgdenghe Ot pO Cem thagess, ents? Muwo & 009s: Seen tee ——- 
hee Sate ole? ol po ohlerkg tata ee be feerene as ewoe seete cevagae Drnreveuteermecses? Ta | eal A © mesae 4 Ee brits geue imate at Vaeess Rate beh & fact at & re pie A To haere ds Not ae Gina, tty be enteend mesure ee mene op tea ar hedintgeomer blip tye an ee 
Pe ode 008, “Vet teed, greg osgteee 8 Shes se A cammg Poles Ohh: F8SigbeQaca Bree eqhg oP F a% cahagpele i hte ee oe pe — , = —e ° Senne 
Eh BOSD See eAFeMhe ace. cies 2 tt heee pee ‘ + ee geage ‘ A ovdepgne * © ene Ord Wee, <O8CTR Povengee hehe cpmretm, Te Seber eaPP OcGaM- awe er © "eR woe Saco wen woe ave = Sry eee eee = 
jamie Whetery °° (te Oh 2 Hue teendses ei, 69,08 FOSS 9 98 Te) merece LAbd Boe Shes ereponseg ee A ° TT a ee eee Pe ed ee ee 2 peg 
5) me Ooh ce en Oo EEE He oO tAe ead te a . ovew « dw a 5 ‘os #0 +N ° rs “A OPPO Ral ofl Oo, My L gga SS & EPO -piQl Oo OO: Bae ww, os . 
thay pean: Lebhadtlyses Litks) thi Mote ore e ; aoe ome oF i) od ea = tostacedted otece a caten vise ‘ oem Prov CDOs SRadgarae ete OEd Se O- a Se: 3.0 0,00! ty bpe ty? 0, te pe fray oI * 
2 Shed" Saree tev ee sb: sanwes brah eane ‘aieeaeen + Gon of DY cede ian @ © ol " 5 ' seen HRs: 046 Ge pBE wd Betavren® FoF e% a oBPr Oe Ph Cede? 99, Code hr eceuty CHa © pSe o,0 Phe" oreteaee Seeee ve z e 2 Ome ee WO 8 i et CD, CeO ® 4, A.D, eer ay? ees 
PW Mtoe een ee Out a. € ewate ntges *@ erset 5g 2 Gee ce bre ' 8 a 8 onjoed 540 Sete h Seger Ohm Fe Le yrRP Rt ete wed RE OP Gas 1c DHS 056.8 + MOS © AES” ne hate aptenle ica 
Td reeneecsanmnslasitgs atone She we 8a its tee Scagme-cde & yee - a 7. cae e <G a cncgeumy sae eee c= 3g tn Mie te rea ¢ ¢ tad Pept ‘ PO eTOn® WELLS Bera crs ModeRes{Baeee Or © fmceh aw cele ° neat 
5 o%o we Tele *Werter, oiah ee Ogee © § 800 anos ape cdAdeh Per Qee ers wel! he ot ous *Sembhse aman enn eete ome tapes. a Sond ~ .= etpeetet = eh eres Se ps 
seregritepem Calpe eset © ow . & 2% 8 Che be owe coe a Pe ee ey JO SORE: P tes) bees ote a ae 
en P te ecetse Sex . = r 8 ‘ ee ce Anebes e Gallet 8 Tt Pe ¢ Dye Me ere = a Op, 2, toe Bumy=se 20 7g eo Angee he ememaienaed bo ~, 
pram eo ot # «Bi greee ae Pas an ee - yes sep ca 0tiie aoe ai B= = : AG cane hr pede sat “ be meee: oy ee ot senveacanteOl peslseemiee Migs ws 6.0 Ase be dB Ome UUNee Leahy bn: pa ert e oe —— 
MOVs [Pn0 re obesee Ce0spe 9g eUe Steed © = Bay San 3 CeHi gh Meg? eee Vale raaehne » JO 20 U epyoos Sete ree + pe: Ot oH oe SENOROS Be tet aP MR bad ge Pe o-aNg OOM, & Orme EtQenes nergy Apple Le (Tatas adig hyped pac in Pe vee : 
baling lL eodeched ba tie MS c0hsbee = wh ny aBegn® cold © © = 26Pe AEF Apaphe cope§, 29.2 TO BL. 0 Oy N Sgro thOumerty ognrgmrend jes mend>te 2B ,dne- "de 88H O10 .As @ Me athe tye Au: Sogneete papa hee 
A 4 Podibecacas ee tal estes an et © ep nO 0! On Pot Oe Coy meek e a 7 age 20 hee ee & oD @ ey fF OEDEh «=o eP1eTY ODE Edberniptgtors Os Whee ofa Seaae oN i Oe eter arm aun ia Se iia evantanges 
. . © Oat Peas s See Segoe: #0 “4 i) my Octet 5 8@ 3 eet Y 2M cots teh se in *s, Be Otsap oder: Bene 81 Bey eo. h = 
485. Red PPS PAA Me iis Sin eee ° Secale WA el ad vet ebese o He Qki Weer 8 Rett eet eaty ve Sele l teak Primary ter ade ret «te Bs Wa OtBe vae— 0er Oo mums © ae (pty On taay OVI He BNeF- O99 810-8} on tirella bapaeety Aa % 
Ae ~Lq pes OM ps Fabre Baie or 08s PP um emg sd ode gOree c bie: ali come amy gestSPhee 8 ft oy ee & Pete e8Ee Me che beer EtG 5 ete Mt nte ER Me 9G ts by tehe tQDOOe rae meds Cr Oe Ke, | Wp tO Be oe RR aS AFR SE] VAST G GPS See os 





OD Eve ye Se - 2 . a re ot) 2 
i. DREN ere r Gs he Pe yea | fer Aree ones tae ge rkiw Mtate eat? och mnceweta Waste ¢ mm cel actus Abt e feo  — Suddee SUP OSagte obs a= 8 -qroades Of + Mey * Oto prs 0) tedye ec OP Pee 2 OD we Meno ytad ante e eh eee ae 5 duiuanh neennaly @arth OSORNO 








